Goals Types for Goal-Oriented Scenarios

In a goal-oriented scenario, you define the goals you want your test to achieve and LoadRunner automatically builds a scenario for you based on these goals.

This section discusses the types of goals for a goal-oriented scenario.

In this topic:

• Virtual Users
• Pages per Minute/Hits per Second/Transactions per Second
• Transaction Response Time

Virtual Users

This goal tests if your application can run a specified number of Vusers simultaneously. Running this type of goal-oriented scenario is similar to running a manual scenario.

 Back to top

Pages per Minute/Hits per Second/Transactions per Second

These goals test the strength of your server. For each of these goal types, you specify a minimum-maximum range of Vusers for the scenario to run, and in the case of the Transactions per Second goal type, you also specify a transaction name.

Note:

• Pages per Minute and Hits per Second goals are for Web Vusers only.
• Hits per second relates to HTTP requests per second.

When you define one of these goal type, Controller divides the target defined by the minimum number of Vusers specified, and determines the target number of hits/transactions per second or pages per minute that each Vuser should reach.

Controller then begins loading the Vusers according to the load behavior settings you defined, as follows:

• If you selected to run the Vusers automatically, LoadRunner loads 50 Vusers in the first batch. If the maximum number of Vusers defined is less than 50, LoadRunner loads all of the Vusers simultaneously.
• If you chose to reach your target after a certain period of the scenario elapses, LoadRunner attempts to reach the defined target within this period of time. It determines the size of the first batch of Vusers based on the time limit you defined and the calculated target number of hits, transactions, or pages per Vuser.
• If you chose to reach your target by gradation (x number of pages/hits every x amount of time), LoadRunner calculates the target number of hits or pages per Vuser and determines the size of the first batch of Vusers accordingly. (Not relevant for the Transactions per Second goal type).

After running each batch of Vusers, LoadRunner evaluates whether the target for the batch was achieved. If the batch target was not reached, LoadRunner recalculates the target number of hits, transactions, or pages per Vuser, and readjusts the number of Vusers for the next batch to be able to achieve the defined goal. By default, a new batch of Vusers is released every two minutes.

If the goal has not been reached after Controller has launched the maximum number of Vusers, LoadRunner attempts to reach the defined target once more by recalculating the target number of hits, transactions, or pages per Vuser, and running the maximum number of Vusers simultaneously.

A Pages per Minute or Hits/Transactions per Second goal-oriented scenario is assigned a Failed status if:

• Controller has twice attempted to reach the goal using the maximum number of Vusers specified, and the goal could not be reached.
• No pages per minute or hits/transactions per second were registered after the first batch of Vusers was run.
• The number of pages per minute or hits/transactions per second did not increase after Controller ran a certain number of Vuser batches.
• All the Vusers that ran failed.
• There were no available load generators for the type of Vusers you attempted to run.

 Back to top

Transaction Response Time

This goal tests how many Vusers can be run simultaneously without exceeding a desired transaction response time. You specify the name of the transaction in your script that you want to test, and a minimum-maximum range of Vusers for LoadRunner to run. The transaction response time you specify should be a predefined threshold value.

For example, if you do not want a customer to wait more than five seconds to log in to your e-commerce site, specify a maximum acceptable transaction response time of five seconds. Set the minimum and maximum number of Vusers to the minimum-maximum range of customers you want to be able to serve simultaneously.

If the scenario does not reach the maximum transaction response time that you defined, your server is capable of responding within a reasonable period of time to the number of customers you want to be able to serve simultaneously. If the defined response time is reached after only a portion of the Vusers has been executed, or if you receive a message that the defined response time will be exceeded if Controller uses the maximum number of Vusers defined, you should consider revamping your application and/or upgrading your server software and hardware.

Note:

• To achieve a Transactions per Second or Transaction Response Time goal, your script must contain transactions. For each of these goal types, you define the transaction in the script that you want to test.
• For a Transaction Response Time goal-oriented scenario to be effective, you must choose your transaction carefully, ensuring that it performs effective hits on the server.

Manual Scenarios

You build a manual scenario by selecting scripts to run, assigning load generators on which to run the scripts, and distributing Vusers to run among the scripts.

You can design a manual scenario in one of the following modes:

• Vuser group mode. In this mode, each script you select for the scenario is assigned to a Vuser group. You assign a number of Vusers to each Vuser group that you create. You can instruct all Vusers in a group to run the same script on the same load generator, or you can assign different scripts and load generators to the various Vusers in a group.
• Percentage mode. In this mode, you define a total number of Vusers to be used in the scenario, and assign load generators and a percentage of the total number of Vusers to each script.

After you define which Vuser groups/scripts to run in the scenario, you select or build a schedule by which to run the scenario. For more information, see Scheduling Manual Scenarios.

You can also create Service Level Agreements (SLAs) which are specific goals that you define for your load test scenario. When you run the scenario, LoadRunner gathers and stores performance-related data. When you analyze the run, Analysis compares this data against the SLAs and determines SLA statuses for the defined measurements. For more information, see Service Level Agreements.

Changing Scenario Modes

You can convert a scenario from the Vuser group mode to the percentage mode and vice versa.

The following table describes what happens to the scenario when converting from the one mode to the other:

Vuser group mode to percentage mode

If a Vuser group contains multiple scripts, in percentage mode the scripts are listed one by one in the Scenario Scripts pane. In the percentage mode, all load generators are assigned to all Vuser scripts by default. If multiple load generators are assigned to a Vuser group, the Vusers assigned to the scripts in the percentage mode are distributed evenly among the load generators originally assigned to the group. If you defined group schedules for the Vuser groups, these settings will be lost. All profiles will contain schedule by scenario settings only. For details about scheduling scenarios, see Scheduling Manual Scenarios.

Percentage mode to Vuser group mode

Each script is converted to a Vuser group. If you defined multiple load generators for a Vuser script, the Vuser group that is created when converting the scenario will also contain multiple load generators.

How-to run JMeter test in LoadRunner / Performance Center 12.55?

Starting with Micro Focus LoadRunner (LR) 12.55 and Performance Center (PC) 12.55, you can execute JMeter tests in addition to other LoadRunner scripts. A scenario containing a JMeter test is supported for both local and remote execution, regardless of operating system.

Results from the LoadRunner scripts and JMeter test are collected and displayed, in a single location during the scenario execution, and are then available for investigation in Analysis.

The best part is that all of this is currently free and unlimited!

This blog post will show how to run JMeter scripts from LoadRunner Controller

Prerequisites:

• Apache JMeter installed on each load generator machine running JMeter tests.
• Java installed (as recommended by Apache JMeter):
  • Windows – PATH should include the jvm.dll directory
  • Linux – LD_LIBRARY_PATH should include the libjvm.so directory
• Environment variable (recommended):
  • JMETER_HOME – pointing to the Apache JMeter base folder
  • JAVA_HOME – pointing to the installed Java

Setting for LoadRunner scenario with a JMeter test:

1. Open LR Controller
2. Select the JMeter Scripts radio button
3. Press the Browse… button
4. Select JMeter Test file (e.g. Test_1.jmx) and click Open
   
   
5. Click OK button on the next window
   
   
6. Now you have JMeter tests in addition to LoadRunner scenarios.
   
   

Note:

• The JMeter test will run as configured in the jmx file. The thread groups and internal scheduling will be                 handled by JMeter.
• For JMeter groups, each virtual user will run a JMeter instance. If your jmx file is configured to execute 1000        threads and the group is configured to run 2 virtual users, 2 JMeter processes will run and 2000 threads will         be executed.
• The JMeter test will be copied to temporary location on the Load Generator and will not affect the origin.

 Runtime Settings for a JMeter test:

Runtime Settings provide general settings for executing the JMeter instance, regardless of the test content.
Select the JMeter test in the Scenario Groups and press  (or Alt + t) to open the Runtime Settings.

• Start measurements check box (checked by default)
• JMeter path (JMeter home directory on local or remote load generator).
  If it is not set, LR uses the JMETER_HOME environment variable.
• JMeter port defines the JMeter port management range:

These ports are used for local communication between the load generator agent and the JMeter process.

• Default: 4000 to 65535
• Range: by user define

Measurements for JMeter test:

We have developed a Backend Listener to receive online measurements from JMeter tests. These measurements are configurable in Runtime settings. When enabled, the graphs are available under a new section: Available Graphs -> JMeter Graphs. Under this section, the following graphs are available:

• JMeter Active threads
• JMeter Hits
• JMeter Throughput
• JMeter Transactions
  
  

Note: The best practice is to use online measurements when creating and testing the load test. To improve performance, Apache JMeter recommends turning online measurements off for real load tests (while no data will be collected from JMeter by Controller)

Analyze your results in Analysis:

All of the data from the test run, for both – LoadRunner scripts and JMeter tests are available in Analysis. Here you can analyze the tests results, compare graphs data and cross correlate results between scripts/rests.  

Best practices:

1. Execute JMeter tests on a remote load generator.
2. Use the latest version of Apache JMeter (version 2.13 and up are supported)

Troubleshooting:

If you experience trouble executing your JMeter test, check the following items.

Case	Option 1	Option 2
Error: “Cannot find the JMX file 'C:\jmeter_tests\test.jmx‘”	The file is not present in this location	Not enough disk space on load generator to clone the file.
No Graphs exists in Controller neither Analysis	The “Start measurement” check box is not enabled
Error: “The JMETER_HOME environment variable is not configured.”	Need to define the JMETER_HOME environment variable	Need to define the JMeter path in the Runtime Setting
Error Pattern: Message contains “Problem loading XML from: ‘……’ missing class ….”	Sometimes JMeter test contains 3rd party plugins. And those plugins are missing in the current used JMeter installation.	Remove the unnecessary 3rd party plugins from the JMeter test Add those 3rdparties’ dependencies to the current used JMeter
“Unable to bind to free port (for Shutdown/ StopTestNow) in range 4445 - 4455. Please extend the port range in the JMeter Runtime Settings. “	In Runtime Setting change or extend the range	In Runtime Setting change the port range radio-button to default

A-Z of Capacity Management: Practical Guide for Implementing Enterprise IT Monitoring & Capacity Planning

Most often we are told the "what and why" of capacity management, but not how to make it happen. This book provides good practical approach on how to implement the process, with a view to bringing its benefits to the organization. Capacity management is incomplete without business driven capacity planning

what is capacity management?

Capacity management is the IT risk management process for ensuring there is adequate infrastructure and computing resources to meet the current and future demand of the business in a cost effective and timely manner
This blog provides a detailed guideline for practical implementation of the capacity management process. The full benefits of implementing the process is usually harnessed when it operates as a value-added process - where capacity planning is driven from business demand, and not just based on infrastructure resources utilization, like CPU, memory, etc. 
Covers:

• Capacity management goals, benefits and strategy

• Common errors and gap analysis guideline

• Identify business metrics and collection techniques

• Implementing CDB - data aggregation and storage

• Capacity planning guideline, and how to build capacity planning model

• Creating capacity plan and stakeholders review

• Capacity Management Report-audience

• Auditing the capacity management process

• Capacity management in cloud computing and machine learning era.

Each chapter has thought provoking questions you may need to ask about the current or planned implementation of capacity management process in your firm. 
Anyone interested in delivering excellent service to customers, but with focus on reducing IT Infrastructure spending will find this book very rewarding.

Loadrunner SOAP over JMS script for Websphere MQ

Prerequisite

1. JMS Queue details
2. Websphere MQ Client installed (if not installed, instructions are below)
3. Websphere MQ jar files installed (if not installed, instructions are below)
4. JDK installed (if not installed, instructions are below)

JMS Queue Details
Depending on which queue(s) you want to send and receive a message from as well as the MQ architecture design, you will require following MQ information from your Websphere MQ Admin.

• HostName
• Channel
• Port
• Queue Manager
• Input Queue
• Output Queue
• Queue Connection Factory
• Username and password

For example, in my case, to send a message to an Input Queue, following information was required:

• HostName – xxx.xxx.xxx
• Channel - ICF.DEF.SVRCONN
• Port - 1414(might be different)
• Queue Manager - Not Required
• Input Queue - Input_queue_name
• Output Queue - Not Required
• Queue Connection Factory - qcf
• Username and password - Not required

Webshere MQ Jar files

1. Install JAVA JDK on your local machine
2. Copy MQ jar files into the JDK...->Java ->jre-> ext folder. You will need to get these jar files from your Webpshere MQ admin
3. Also, since we are using fscontext as initial context factory, you will need to download and save fscontext and providerutil.jar files in the above mentioned folder

How to create JNDI binding

1. Install Websphere MQ Client application on your local machine.
2. Navigate to the location where JMSAdmin.config file is located and open it. We are going to select a context factory to use and path where to create the binding file. In my case this file is located at C:\Program Files (x86)\IBM\WebSphere MQ\Java\bin Update and save the file with following details:
INITIAL_CONTEXT_FACTORY=com.sun.jndi.fscontext.RefFSContextFactory PROVIDER_URL=file:/C:/JNDI


3. Create a new qm.scp file (make sure this file is in the same folder as JMSAdmin.bat file) and put there the MQ details. The .scp file will look like this:
DEFINE QCF(qcf) tran(client) chan(ICF.DEF.SVRCONN) host(xxx.xxx.xxx) port(1414) DISPLAY QCF(qcf) DEFINE Q(Input_queue) QUEUE(Input_queue) DISPLAY Q(Input_queue) end


4. Create the JNDI folder on your C drive. This is where your binding file will automatically be saved.
5. Navigate to “...\WebSphere MQ\Java\bin” and edit JMSAdmin.bat by replacing “java” text with the full java path incase it is not already defined in your System environment settings.
6. Navigate to Websphere MQ Java bin folder via the command prompt and execute the JMSAdmin Tool (JMSAdmin. Bat file) with qm.scp as the parameter. This will generate a .bindings file in the JNDI folder automatically.





You will see something like this when JMSAdmin bat application is executed.


Cool, JNDI binding is done.

Creating a Webservices Loadrunner Script

1. Open up a new web services script.
2. Press F4 to Navigate to Run-time setting and update the following fields in JMS->Advanced option:
-JVM Home: JAVA path
-JNDI initial context factory: com.sun.jndi.fscontext.RefFSContextFactory
-JNDI provider URL: file:/C:/JNDI/
-JMS connection factory: QCF name


3. Now you are ready to create your MQ script.
4. You can then use Web Services jms_set_general_property, jms_send_message_queue, jms_set_message_property and jms_receive_message_property functions.

NOTE: JMS Bindings Extensions are not supported in Loadrunner 11.

How to configure Dynatrace with Loadrunner 12.55?

Dynatrace Monitor

The Dynatrace monitor provides information from the Dynatrace server about monitored applications.

In this topic:

• Dynatrace dashboards and dashlets
• Set up the Dynatrace monitor

Dynatrace dashboards and dashlets

Using Dynatrace's dashboards and dashlets you can get insights into your full application stack, in the form of deep transaction tracing, synthetic monitoring, real user monitoring, and network monitoring.

A dashboard represents a specific view of one or more Dynatrace data sources. A dashboard is comprised of dashlets, the building blocks of a dashboard that contain specific types of data (in the form of a graph or data tables).

Note: Only dashlets that are represented as graphs in Dynatrace are available in Performance Center during the performance test run.

 Back to top

Set up the Dynatrace monitor

1. Prerequisites: The Dynatrace monitor must be installed on the AUT server.
2. In Controller, in the Run tab, add the Dynatrace monitor. Available graphs > Dynatrace graphs > Dynatrace.
3. Right-click the Dynatrace monitor and click Add measurements.
4. In the Dynatrace dialog box, click Add to add the monitor, and enter the Dynatrace server details:
   Note: In order to connect to a Dynatrace server with specific credentials, you must first stop the Dynatrace monitor and remove all of its counters.

   UI Element	Description
   Name	The name of the Dynatrace server.
   Port	The Dynatrace port. Default: 8021
   Use Secure HTTP	Uses a secure HTTP connection to the Dynatrace server.
   User Name	Instructs LoadRunner to use the name of the Dynatrace account.
   Password	Instructs LoadRunner to use the password for the Dynatrace account.

   Click OK.

For detailed information on the performance measurements that Dynatrace can monitor, refer to the Dynatrace documentation.

Load Runner Tips and Tricks

1. Consult your developer if you need help to identify the protocol used by the application.

2. Some other applications, such as antivirus, VPN’s, firewall, and so on may interfere with the recorder. In case of problems, always look to see what is installed on the machine and try to disable software that looks like it is interfering with socket level data.

3. Delete (or comment out) all the “web_add_cookie” statements in the script. Some applications use explicit cookies to keep track of users. Therefore cookies statements have to be commented out otherwise old cookies are sent to the server again and it will not work.

4. Delete all the web_url (…..) statements in start of the script, which are downloading the windows updates or something like that.

5. If you are using the Multiple Protocol recorder, you can use this option of Tools →“Regenerate Vuser” to regenerate the script, if the earlier modified script is not working fine.

6. Sometimes, you will encounter cases where the Web recorder (Single and Multi) records some initial login steps but nothing else after that, even though it is still using the HTTP protocol. This can happen if the application is dependent on the cache in some way. By default, LoadRunner recording will disable the cache.

To set LoadRunner not to interfere with the cache setting, change the following in registry:

a. Go to Start → Run, and enter regedit to open the registry editor.

b. Navigate to HKEY_CURRENT_USER\Software\Mercury Interactive\ LoadRunner\ Protocols\ HTTP\Analyzer

c. Set AddNoCacheHeaderToHtml=0.

7. The following are limitations of the Single Protocol Web recorder. If any of the following is causing a problem, you are recommended to switch to use the Multiple Protocol web recorder to record.

Note: All the issues below can be resolved with Multiple Protocol Web recorder.

a. While recording, the Single Protocol Web recorder will redirect the client requests to a local proxy, at localhost: 7777. For such, make sure that you have the permission to change the LAN settings of the machine.

To verify:

i. Open Internet Explorer and go to Tools → Internet Options →Connections → LAN Settings.
ii. Make sure that you can modify the entries here.

b. The Single Protocol Web recorder obtains the proxy setting from the HKEY_CURRENT_USER registry hive. Verify the following to make sure that it is set up correctly:

i. Go to Start → Run, and enter regedit to open the registry editor.
ii. Navigate to HKLM\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\InternetSettings.
iii. Verify the non-default key, ProxySettingsPerUser. If the
iv. ProxySettingsPerUser value exists and has a value data of 0 (zero), set it to

1. It is okay if you do not have that registry key. As long as ProxySettingsPerUser is not set to one, proxy settings will be obtained from the HKEY_CURRENT_USER registry hive.

1. To make permanent change in default settings in VuGenerator:

To keep from having to change the run-time settings every time that you create a new script, here is an easy way to save your own default settings.

1. Create a new script

2. Make the setting changes that you need to make.

3. Save the script

4. Now, copy the default.cfg file out of the directory of the script file that you just created to the Program Files\Mercury Interactive\LoadRunner\template\{dir}.

The {dir} is whatever directory that you are creating your scripts from. For example, a Web/HTML vuser directory is the \qtweb\ directory. You can also change the init.c, end.c, or action.c if you seem to always have to make changes to these files every time you create a new script.

Set up LoadRunner to run nightly

To get LoadRunner to run nightly required two things. The first was a batch file that would execute LoadRunner and then invoke the analysis and the second was how to schedule the execution on the controller.

The batch file to run LoadRunner at night and to save the results in a specific directory for that nights run is shown below:REM Batch to run nighlty performance test REM REM Thanks to Simon Shepard at http://www.ss64.com for date code REM Calculate today date FOR /f "tokens=6-8 delims=/ " %%G IN ('NET TIME \\%computername%') DO ( SET _mm=%%G SET _dd=%%H SET _yy=%%I ) REM Run LoadRunner "C:\...\bin\Wlrun.exe" -Run -TestPath "P:\Projects\...\PeakHourSmall.lrs" -ResultLocation "P:\...\Results" -ResultCleanName Night-%_dd%-%_mm%-%_yy% REM Run analysis "C:\...\bin\AnalysisUI.exe" -RESULTPATH "P:\...\Night-%_dd%-%_mm%-%_yy%\Night-%_dd%-%_mm%-%_yy%.lrr"

To get the batch file to run as a scheduled task took longer than expected due to some restriction on the PC at work. To overcome this I had to use schtask.exe to schedule the batch.schtasks.exe /create /SC daily /ST 01:50:00 /TR "P:\...\Nightly.bat" /TN RunPerfTest

The nightly LoadRunner test now runs at 1:50 in the morning and the results are waiting when I arrive in the morning.

Saving results of SQL query to a LoadRunner parameter

Here is an example of how to save the results of a query to a loadrunner parameter. In this example I want to know the prod_code for a particular user whos user name is held in the parameter pUserID. To do this you have to add a lrd_ora8_save_col statement BEFORE the fetch statement.

lrd_ora8_handle_alloc(OraEnv1, STMT, &OraStm7, 0);
lrd_ora8_stmt(OraStm7, "select indiv,prod_code,last_up_seq from product_users"
"where indiv_uid = '{pUserID}'", 1, 32, 0);
lrd_ora8_exec(OraSvc1, OraStm7, 0, 0, &uliRowsProcessed, 0, 0, 0, 0, 0);
lrd_ora8_bind_col(OraStm7, &OraDef9, 1, &INDIVD10, 0, 0);
lrd_ora8_attr_set(OraDef9, CHARSET_FORM, "1", -1, 0);
lrd_ora8_bind_col(OraStm7, &OraDef10, 2, &PROD_CODE_D11, 0, 0);
lrd_ora8_attr_set(OraDef10, CHARSET_FORM, "1", -1, 0);
lrd_ora8_bind_col(OraStm7, &OraDef11, 3, &LAST_UPE_SEQ_D12, 0, 0);
lrd_ora8_save_col(OraStm7, 2, 1, 0, "prod_code");
lrd_ora8_fetch(OraStm7, -1, 15, &uliFetchedRows, PrintRow8, 2, 0, 0);

lrd_handle_free(&OraStm7, 0);

lr_message("The users product code is %s",lr_eval_string("{prod_code}"));

The second and third arguementare the column and row number with the 1,1 returns the value in the first column and the first row, The value is stored in the last arguement. Note that if no value is returned the error will be an unhelpful Parameter is not initialized.

LoadRunner Analysis Importing PerfMon Data

I have recently imported some perfmon data into LoadRunner analysis. First of all I had to use perfmon’s relog command to convert the data into a CSV file.

I then imported this using Tools -> External Monitors -> Import Data. Which leads you to the following dialogue box.



I think this is because I had selected the Time Zone as . So I reloged the data to match the start and end time of the LR results and imported again with the Time Zone set to 

Expanding LoadRunner functions

I quickly wanted to get some lr_status message outputted to the console for every LoadRunner transaction in a script, I basically wanted to insert a status message every time the lr_start_transaction was called to output the transaction name to the console. however, it was a large script and I didn’t want to make any code changes to the load runner script. So I wanted to expand the existing function. The code below was added into the vinit section.

void my_start_transaction (char *name)
{
lr_vuser_status_message("Starting Transaction %s",name);
lr_start_transaction(name);
return ;
};
vuser_init()
{

#define lr_start_transaction(x) my_start_transaction(x)

return 0;
}


LoadRunner Function Override

Creating LoadRunner Dynamic Transaction Names

Today I wanted to use parameterized transaction names within a LoadRunner script. As I thought that a particular transaction was failing after a particular number of iterations. Luckily this was pretty simple in LoadRunner. It was just a case of using lr_eval_string in the call to the transaction wrapper.

lr_start_transaction(lr_eval_string("Do Something {pIteration}"));
…
lr_end_transaction(lr_eval_string("Do Something {pIteration}"), LR_AUTO);

You have to be careful above to make sure that start and end transaction names are the same. To overcome that problem I created a string variable to hold the LoadRunner transaction name.

char sTranName[20];
…
sprintf(sTranName,lr_eval_string("TransactionA_{pIteration}"));
lr_start_transaction(sTranName);
…
lr_end_transaction(sTranName,LR_AUTO);