3 Replies Latest reply on Sep 4, 2014 1:36 PM by jameswright

    Have you heard of the jq command line tool for JSON parsing?

      I just discovered this awesome tool today, and wanted to take a moment to share it with everyone   The tool is called jq.

       

      To get it going, I used Homebrew for Mac, with a simple:

      brew install jq

       

      In my scenario I was trying to do some manipulating of securityGroup members, so I was using:

       

      After some reading of the jq Manual, I was able to construct the following command:

      curl -u $user:$pass -X GET -H "Content-Type: application/json" "https://your.jiveon.com/api/core/v3/securityGroups/$i/members" | grep -v 'allowIllegalResourceCall' | jq -r '.list[].resources.self.ref'

       

      In essence, this command is:

      • connecting to my Jive Instance
      • getting a list of members for a security group
      • stripping the throw declaration
      • then parsing the JSON output
      • looking at the list property in the output and return all values resources.self.ref that exist in the items in the list array
        • the -r argument strips the quotes from the output (see below)

       

      Raw V3 API Outputjq Output

      throw 'allowIllegalResourceCall is false.';

      {

        "itemsPerPage" : 25,

        "list" : [ {

          "id" : "1234",

          "resources" : {

            "trendingPlaces" : {

              "allowed" : [ "GET" ],

              "ref" : "https://your.jiveon.com/api/core/v3/people/1234/@trendingPlaces"

            },

            "trendingContent" : {

              "allowed" : [ "GET" ],

              "ref" : "https://your.jiveon.com/api/core/v3/people/1234/@trendingContent"

            },

      ....

      "https://your.jiveon.com/api/core/v3/people/1234"

      ...

       

      (note:  If you add the -r parameter to the command above) the result is:

      https://your.jiveon.com/api/core/v3/people/1234

      ...

       

      As you can see, this is a pretty simple way to traverse JSON and bring the power of the V3 API to shell scripting, especially some of these example: REST API v3 Examples

      What I also love is that there is a free live parsing service for this tool called: jq play

      This tool lets you put in some JSON and iterate on the proper jq syntax to get the output you want! ftw


      Share Your Script on GitHub 

      Lots of interesting tools ideas I have here, and going to share some of these scripts in this GitHub repo:

      jivesoftware/scripts · GitHub

       

      What do you think

      Do you have some cool shell scripts that you use that you'd like to share?  Remember, contributing this repo constitutes points in the

      From Points to Pints in the 2014-15 Jive Developer Program

       

      Looking forward to the conversation, and hope this little find helps everyone =)

       

      Sean Horn James Wright Brett VanderHaar LG . - This seems right up your ally =)

        • Re: Have you heard of the jq command line tool for JSON parsing?
          jameswright

          Ryan,

           

          Thanks for sharing. JQ looks very helpful.

          I actually use a tool called BaseX and the XQuery language for this type of thing. Its another great option to be aware of. Ill add these scripts to the repo if you would like when I have some time.

           

          For now, here is all you need to get started: BaseX | The XML Database

          1) brew install basex

           

          Once installed, launch the BaseX GUI.

           

          Install the jive.xqm module attached to this post which I wrote to take care of some of the basic REST stuff: (Its only 70 lines of code). Here are the steps for installing the package:

           

          2) In the BaseX GUI navigate to Options -> Packages -> Install...

          3) Then select the jive.xqm file attached to this comment.

          4) Click Open

           

          Here are the methods included with that module:

           

          request-template(user, pass)

          get-item(request, uri)

          get-all-items(request, uri)

          update-item(request, item)

          create-item(request, uri, item)

          delete-item(request, uri)

          invite-to-group(request, emails, group)

           

          Once the module package is installed, you can include it in future scripts. Here are a few quick examples:

           

          Ryans example using the XQuery approach:


          import module namespace jive = 'http://seu.jive.com';

           

          let $destination := 'https://communityUrl.com'

          let $apiUri := '/api/core/v3/securityGroups/$i/members'

          let $destReq := jive:request-template('$user', '$pass')

           

          for $item in jive:get-all-items($destReq, $destination || $apiUri)

             return $item/resources/self/ref

           

           

          Delete all threads with images:

           

          import module namespace jive = 'http://seu.jive.com'

           

          let $baseURI := 'https://communityUrl.com'

          let $request := jive:request-template('$user', '$pass')

          let $content:= jive:get-all-items($request, $baseURI || '/api/core/v3/contents?count=100')

          return

            for $thread in $content return

              let $images:= jive:get-all-items($request, $thread/resources/images/ref) return

              if(count($images/list/*) > 0) then (

                    jive:delete-item($request, $thread)

           

              ) else ()

           

          Delete multiple users with the 'gis' tag applied:

           

          import module namespace jive = 'http://seu.jive.com';

           

          let $baseURI := 'https://communityUrl.com'

          let $request := jive:request-template('$user', '$pass')

          let $usersToDelete := jive:get-all-items($request, $baseURI || '/api/core/v3/people?filter=tag(gis)') return

          for $user in $usersToDelete return

               jive:delete-item($request, $user/resources/self/ref)

           

          Move all the content with a tag to a specific space:

           

          import module namespace jive = 'http://seu.jive.com';

           

          let $communityURI := 'https://communityUrl.com'

          let $request := jive:request-template('$user', '$pass')

          let $contentQuery := $communityURI || '/api/core/v3/contents?filter=tag(sometag)'

          let $newSpaceURI := '/api/core/v3/places/23232' return

          for $content in jive:get-all-items($request, $contentQuery ) return

            copy $copy := $content

            modify (replace value of node $copy/parent with $newSpaceURI)

            return $copy

                 jive:update-item($request, $copy)

           

          Invite a user to all groups with a specific tag:

           

          import module namespace jive = 'http://seu.jive.com';

           

          let $destination := 'https://communityUrl.com'

          let $apiUri := '/api/core/v3/'

          let $destReq := jive:request-template('$user', '$pass')

          let $type := 'places?filter=tag(gis)'

          let $invite :=

            <json type="object">

              <body>Please join the client group</body>

              <invitees type="array">

                <_>https://communityUrl.com/api/core/v3/people/4194</_>

              </invitees>

            </json>

           

          for $item in jive:get-all-items($destReq, $destination || $apiUri || $type )

             return 

               jive:create-item($destReq, $item//invites/ref/text(),  <http:body media-type="text/plain">{json:serialize($invite)}</http:body>)

           

           

          Create a CSV file for all the users that have been give access to a document which was shared by a specific user:


          import module namespace jive = 'http://seu.jive.com';


          let $destination := 'https://communityUrl.com'

          let $destReq := jive:request-template('$user', '$pass')

          let $apiCall := '/api/core/v3/contents?filter=author(https://communityUrl.com/api/core/v3/people/2023)'

          return

            let $csv := <csv>

              {

                for $item in jive:get-all-items($destReq, $destination || $apiCall )

                   where $item/users

                   return

                   let $record := $item//users/.. return

                   <record>

                     <documentName>{$record/subject/text()}</documentName>

                     <sharedBy>{$record/author//name/formatted/text()}</sharedBy>

                     <users>{$record/users//name/formatted/text()}</users>

                   </record>

           

              }

            </csv>

            return csv:serialize($csv, map { "header": "yes"})

           

          Anyone of these scripts can be augmented to accept command line arguments for use in the console.

          For more information checkout the XQuery language.

          1 person found this helpful