Forum Discussion

dhundley's avatar
dhundley
Regular Contributor
2 years ago

execution plan enabled difference between local and azure

i have an execution plan that doesn't always have every test item/test case enabled. 

 

i came up with the following for loop that (when the plan is running on my local machine) is able to determine which test item is the first and which is the last that will run (because i have special one time only processing that i want to execute in the onstarttestcase and onstoptestcase event handlers before the first test case runs and after the last test case has completed.)

 

this all worked perfectly when i ran the execution plan on my local workstation AND when i performed a remote execution of it on my automation VM using network suite. but after i committed my changes to the azure repo it doesn't work. it acts like every individual test case is the first and last test case. there just appears to be some frustrating differences in testcomplete when running the thing locally and running it from the azure repo especially when it concerns these checkboxes. when running locally i could have the Name check box checked and uncheck the Test Case checkbox and the item would still run but wouldn't show up on a results report. but if i commit it that way the test item won't run in Azure. Variables are another thing treated differently when running from azure. locally even temporary variables maintain their values between test items but on azure only permanent variables do that. 

 

the testitem(index) properties list only seems to have Enabled as a valid property (i.e. the checkbox under the name column of the execution plan). how come there isn't a property for the Test Case checkbox? Can someone provide me with a solution to what i'm trying to accomplish here? How can someone every know if what runs successfully in the Testcomplete IDE is going to also be successful when running from a pipeline when there are these differences? Why even have two checkboxes on the execution plan especially when they seem to behave or have different effects based on where the execution is originating from? 

 

 

 

  • Hi,

     

    I am not Azure guru and might be wrong here, but my understanding is that VS Test Steps component makes it possible to utilize unit tests runner to execute unit tests. And, as per Support, TestComplete Test Adapter is used to provide this runner with a list of tests (that will be executed using unit tests concept). So I think that both these components can be removed if you are going to use Command Prompt task.

    Not sure about e-Mail step - it might or might not work depending on how it is configured. For example, it may stop working if it is set to mail published test result (as the result is published by VS Test Step adapter, if I am not mistaken). But if e-Mail sends result that is prepared by TestComplete itself, than I see no reason why it should stop working.

     

8 Replies

  • dhundley's avatar
    dhundley
    Regular Contributor

    it's also been my experience with running the testcomplete execution plan from azure that the testcase will run if the testcase checkbox was checked when the commit is made even if the enabled checkbox in the name column is not checked. that seems wonky to me. 

    • dhundley's avatar
      dhundley
      Regular Contributor

      i have never used the parameters feature of testcomplete before. is there perhaps some way of accessing the test item's or test case's parameters inside the event handlers? that might be a solution but how do i access the value of the parameter that i put on on the keyword test from within the event handler?

      • dhundley's avatar
        dhundley
        Regular Contributor

        ok. i may have found a solution that is so simple it is jaw dropping. I just want to know if it will still be a solution when it has been committed and is running from azure. instead of using parameters i noticed the Description field on the properties page. I can easily put the test "First" in the Description field of the first enabled test case on the execution plan and "Last" in the Description of the last enabled test case 

         

        i can then set up my event handlers as shown here and it runs successfully in my little test project running on my workstation IDE

        but will it work running from azure? if so, chalk one up for Occam's Razor