Forum Discussion

Stalsy's avatar
Stalsy
Contributor
4 years ago

How to throttle test (get HTTP 429 status code)

I'm using ReadyAPI 3.6 Test.  I need to test all non-200 HTTP status codes.  For 429 (too many requests) I've tried using a data-driven approach with the data source just populating a constant property, then the call to the API and then a groovy script to check and stop the execution if n iterations have been achieved (otherwise the test will loop indefinitely).

But even when I clone the test a couple of times and put it in its own testsuite and run the test suite in parallel mode, ReadyAPI seems to wait till each request returns and I never get HTTP status 429.  Even when we dropped the throttling count to 1 per second, all that happened was that the call just took longer that a second to complete!  So from 120ms average time per call, I got an average of 1.2 seconds so again, no HTTP 429!

  • Stalsy's avatar
    Stalsy
    4 years ago

    Thanks for the quick replies.  It's working now. I think there was an issue with the throttling limit.  To answer your questions:  I did try the load testing but you can't really see the results, this is one of many automated tests that I'm running so I need a clear pass/fail indication.

    When you use a Data Source, you must use a Data Source Loop to get iterations but in my case since I don't know how many iterations will occur before the 429 return code, I set up an infinite loop and the groovy script checks for an early exit if I get a 429.  If I don't get 429 after 100 iterations I consider the test a failure.

    My solution consists of the following:

    1. Data Source step that sets a property

    2. REST request that uses the property

    3. REST request that uses the property

    4. REST request that uses the property

    5. REST request that uses the property

    6. Groovy script to decide pass/fail

    7. Data Source Loop with target the first REST request

    The groovy script checks the current row, and also the Status response headers of the 4 REST requests.

    If the row is less than 100 and if any of the 4 statuses are 429 then the test finishes successfully. If after 100 iterations there is no 429 the test fails.

    Sample code:

    def row = testRunner.testCase.testSteps["Data Source"].currentRow

    if (row <= 100 )

    {
     if (responseHeaders1.get('#status#').toString().contains('429') ||  responseHeaders2.get('#status#').toString().contains('429') ||responseHeaders3.get('#status#').toString().contains('429') ||responseHeaders4.get('#status#').toString().contains('429') )

     {
    testRunner.cancel('Finished Successfully')
    }
    }
    else {
    testRunner.fail('Finished UNSuccessfully')
    }

     

  • Hey Stalsy. If you have a ReadyAPI Test Pro license, you should also have access to the basic functionality of ReadyAPI Performance (for load testing). Have you tried setting up this scenario as a load test to prompt the 429 error? Instead of multiple tests suites, you would setup multiple scenarios with a certain number of virtual users assigned to them. Documentation for load testing is here: https://support.smartbear.com/readyapi/docs/performance/intro/load-testing.html

     

    Also, on a side note, you mentioned that you're using a groovy script to check and stop the loop. There is a Data Source Loop test step that you could also use for this: https://support.smartbear.com/readyapi/docs/functional/steps/data-source-loop.html

     

    Let me know if you have any more luck with the 429 scenario as a load test. 

    • Stalsy's avatar
      Stalsy
      Contributor

      Thanks for the quick replies.  It's working now. I think there was an issue with the throttling limit.  To answer your questions:  I did try the load testing but you can't really see the results, this is one of many automated tests that I'm running so I need a clear pass/fail indication.

      When you use a Data Source, you must use a Data Source Loop to get iterations but in my case since I don't know how many iterations will occur before the 429 return code, I set up an infinite loop and the groovy script checks for an early exit if I get a 429.  If I don't get 429 after 100 iterations I consider the test a failure.

      My solution consists of the following:

      1. Data Source step that sets a property

      2. REST request that uses the property

      3. REST request that uses the property

      4. REST request that uses the property

      5. REST request that uses the property

      6. Groovy script to decide pass/fail

      7. Data Source Loop with target the first REST request

      The groovy script checks the current row, and also the Status response headers of the 4 REST requests.

      If the row is less than 100 and if any of the 4 statuses are 429 then the test finishes successfully. If after 100 iterations there is no 429 the test fails.

      Sample code:

      def row = testRunner.testCase.testSteps["Data Source"].currentRow

      if (row <= 100 )

      {
       if (responseHeaders1.get('#status#').toString().contains('429') ||  responseHeaders2.get('#status#').toString().contains('429') ||responseHeaders3.get('#status#').toString().contains('429') ||responseHeaders4.get('#status#').toString().contains('429') )

       {
      testRunner.cancel('Finished Successfully')
      }
      }
      else {
      testRunner.fail('Finished UNSuccessfully')
      }

       

  • nmrao's avatar
    nmrao
    Champion Level 3
    Is the latency increased after increasing it to parallel?
    If you run a single test now, is latency back to normal?
    • Stalsy's avatar
      Stalsy
      Contributor

      After many runs... no it does not!  I think there was something wrong in the deployment because when I run it today with the throttle set at 90 transactions per 60 seconds, it worked.  I had the test set as a Test Suite to enable the parallel running, but now I can get 429 by just having a loop with 4 identical calls.