1 Reply Latest reply on Oct 12, 2016 9:13 PM by Jeff Shurtliff

    Filtered queries in cURL reporting "Illegal characters found in URL"

    Jeff Shurtliff

      I'm trying to use cURL in a bash script to pull data on all discussions in a specific space in our community.

      When I run the command ad hoc from the terminal it works fine, as shown below.

       

      root@rsalink-api:~/rsalink# curl -G -u adminuser "https://community.rsa.com/api/core/v3/places/3008/contents?filter=type(discussion)&sort=titleAsc&count=9&startIndex=0" > test/testData.json; head test/testData.json
      Enter host password for user 'adminuser':
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      100 84411  100 84411    0     0  52899      0  0:00:01  0:00:01 --:--:-- 52922
      {
        "itemsPerPage" : 9,
        "list" : [ {
          "id" : "186947",
          "resources" : {
            "entitlements" : {
              "allowed" : [ "GET" ],
              "ref" : "https://community.rsa.com/api/core/v3/contents/40909/entitlements"
            },
            "outcomeTypes" : {
      root@rsalink-api:~/rsalink#
      

       

      However, when I try to run the same query in a bash script I get the following error:

      * Illegal characters found in URL

      * Closing connection -1

      curl: (3) Illegal characters found in URL

       

      I have tried the following ways to run the command, which all fail with the error above:

      # Attempt 1
      curl -u $USERNAME:$PASSWORD "https://community.rsa.com/api/core/v3/places/${CURRENTSPACEID}/contents?filter=type(discussion)&sort=titleAsc&count=${PARTIALCYCLE}&startIndex=${STARTINDEX}"
      
      # Attempt 2
      curl -u $USERNAME:$PASSWORD 'https://community.rsa.com/api/core/v3/places/'"${CURRENTSPACEID}"'/contents?filter=type(discussion)&sort=titleAsc&count='"${PARTIALCYCLE}"'&startIndex='"${STARTINDEX}"''
      
      # Attempt 3
      PARTIALCYCLEQUERY=$(echo "${CURRENTSPACEID}/contents?filter=type(discussion)&sort=titleAsc&count=$PARTIALCYCLE&startIndex=$STARTINDEX")
      curl -u $USERNAME:$PASSWORD "https://community.rsa.com/api/core/v3/places/${PARTIALCYCLEQUERY}"
      
      # Attempt 4
      PARTIALCYCLEQUERY=$(echo "filter=type(discussion)&sort=titleAsc&count=$PARTIALCYCLE&startIndex=$STARTINDEX")
      curl -G -v -u $USERNAME:$PASSWORD "https://community.rsa.com/api/core/v3/places/${CURRENTSPACEID}/contents" --data-urlencode "$PARTIALCYCLE"
      

       

      I also ran the command printf %s "$PARTIALCYCLEQUERY" | xxd to make sure that there wasn't an extraneous carriage return and didn't see one.

      00000000: 6669 6c74 6572 3d74 7970 6528 6469 7363  filter=type(disc

      00000010: 7573 7369 6f6e 2926 736f 7274 3d74 6974  ussion)&sort=tit

      00000020: 6c65 4173 6326 636f 756e 743d 3926 7374  leAsc&count=9&st

      00000030: 6172 7449 6e64 6578 3d30                 artIndex=0

       

      Does anyone know what I'm doing wrong or how I need to tweak my commands for them to run successfully in the script?

       

      Thanks in advance!