REST actually doesn't allow explicit export to a certain file type as being stateless is part of the standard. Also, the number of users in some instances can be massive and if multiple people are trying to run the same query, you can see how that can become bad very quickly.
Anyway, this was the first hit in my Google search for shell script JSON to CSV: archan937/jsonv.sh · GitHub
I haven't used it before, but it seems like it fits the bill.
I tried but couldnt get the link you sent to work on the output from my REST call , just black lines output . it works from their sample which was ultra basic.
Having paginated JSON is a pain to work with, at least from shell scripting - is it standard output format from REST APIs in general ?
I guess I am learning that to do anything with our JSON output you need to be in a proper programming language (or be really good a shell / parsing ).
I think I'm going to switch to Python.
Sorry that link didn't work, but I'm glad you found something that would.
Any internet media types are part of the RESTful standard (JSON, XML, HTML), but JSON is the simpler structured data type to work with when compared to XML. Your goal would honestly be easiest with a more sophisticated language but it can be done with either.
I would parse every page and never combine them. With >100k users the file may get bit. With csplit /id/ one can seperate the users into single files and parsing single users with sed/awk should work fine.
Anyhow I'd write a Java program for this as it is easier to debug and maintain.
I ended up doing in Java and shell. Though the shell one is only for smaller sites.
The shell script was to explore how to do it more than being the best language for the job. Idea being a script is usually more universally understood and so more easily adjustable in Sys admin community.
But Java is more fit for purpose here for sure.
Jive Software | Sales Engineer
m 61(0) 488 493799<tel:61(0)%20488%20493799>