Examples on Loadrunner Regular Expressions

I'm going to show and explain how to use Regular Expressions in LoadRunner.

Introduction:
The present article is a summarizing of the LoadRunner Regular Expressions challenge and its results. Also, I added code for RegExp patterns/subpatterns matching.
All LoadRunner Regular Expressions functions are shown with examples.


Outline:

  1. How to check - whether RegExp pattern matches against a text or not
  2. How to get a matched strings (RegExp patterns and subpatterns)

How to check - Whether RegExp pattern matches against a text or not


I thanks Charlie Weiblen and Tim Koopmans for the solution. I modified it slightly.
So, here it is:

  1. Download and unpack Binaries and Developer files for PCRE (Perl Compatible Regular Expressions).
    These and others files are available on Pcre for Windows page.
  2. Unzip downloaded archives into c:\pcreC:\pcre folder
  3. Сomment out the include for stdlib.h file in:
    • C:\pcre\include\pcre.h
    • C:\pcre\include\pcreposix.h
    Commented stdlib.h file
  4. In your LoadRunner script, add to globals.h:
    • #include "c:\\pcre\\include\\pcre.h"
    • #include "c:\\pcre\\include\\pcreposix.h"
    Edited globals.h file
  5. Add the match() function to vuser_init section:

    //////////////////////////////////////////////////////////////////////////
    /// 'match' function matches a 'pattern' against a given 'subject'
    /// It returns 1 for a match, or 0 for a non-match / error
    int match(const char *subject, const char *pattern)
    {
    int rc; // Returned code
    regex_t re; // Compiled regexp pattern

    lr_load_dll("c:\\pcre\\bin\\pcre3.dll");

    if (regcomp(&re, pattern, 0) != 0)
    return 0// Report error

    rc = regexec(&re, subject, 0NULL0);
    regfree(&re);

    if (rc != 0)
    return 0// Report error
    else
    return 1;
    }
  6. Let's run sample LoadRunner script and check the result:
    As you can see, match() function works correctly. Using match() function, you can check - whether RegExp pattern matches against a text or not.

    It can be helpful, when you verify in LoadRunner that the text (RegExp pattern) matches the text on a downloaded page.

    I tested the match() function with different patterns and subject strings:
    #Subject stringPatternsResult of
    match()
    Is correct
    result?
    1abcdefb(c(.*))e1Yes
    2abcdefb(z(.*))e0Yes
    32008\\d{2,5}1Yes
    42008\\d{5}0Yes
    5abc 1st of May 2008xyz\\d.*\\d1Yes
    Note: Since LoadRunner uses ANSI C language, please do not forget to double backslashes (\\). For example, to match any digit character (0-9), use pattern "\\d".

    match() function is simple enough. But it searches only and it cannot extract matched subpatterns from the text. For example, we have to extract the name of month from these strings:
    • "abc 1st of May 2008xyz"
    • "abc 25th of February 2031"
    • etc
    We can use the following pattern:
    • \d.+([A-Z]\w+)\s+\d{4}

    The name of month will be matches by subpattern ([A-Z]\w+). How to extract the found text? You can use matchex() function for that. Let's discuss it in details...
How to get a matched strings (RegExp patterns and subpatterns)

To get a matched (found) strings, we have to update our match() function.
That's why I created matchex() ('match' EXtended) function.
  1. Add the matchex() function to vuser_init section
    //////////////////////////////////////////////////////////////////////////
    /// 'matchex' (EXtended) function matches a 'pattern' against a given 'subject'
    /// It returns number of matches:
    /// 0 - for a non-match or error
    /// 1 and more - for successful matches
    int matchex(const char *subject, const char *pattern, int nmatch, regmatch_t *pmatch)
    {
    int rc; // Returned code
    regex_t re; // Compiled regexp pattern

    lr_load_dll("c:\\pcre\\bin\\pcre3.dll");

    if (regcomp(&re, pattern, 0) != 0)
    return 0// Report error

    rc = regexec(&re, subject, nmatch, pmatch, 0);
    pcre_free(&re); // Release memory used for the compiled pattern

    if (rc < 0)
    return 0// Report error

    // Get total number of matched patterns and subpatterns
    for (rc = 0; rc < nmatch; rc++)
    if (pmatch[rc].rm_so == -1)
    break;

    return rc;
    }
  2. Let's run sample LoadRunner script and check the result:
    matchex() function returns a number of matched patterns/subpatterns and fill an array in with information about each matched substring.


    What is an information about each matched substring?


    This info contains the offset (rm_so) to the first character of each substring and the offset (rm_eo) to the first character after the end of each substring, respectively.

    Note1: 
    The 0th element of the array relates to the entire portion of string that was matched.
    Note2: Subsequent elements of the array relate to the capturing subpatterns of the regular expression.
    Note3: Unused entries in the array have both structure members set to -1.

    Let's investigate it with the example. This is our subject string:
    ExampleThe replay log shows offsets for matched substrings:
    • Action.c(7): Matched 3 patterns
    • Action.c(10): Start offset: 1, End offset: 6
    • Action.c(10): Start offset: 2, End offset: 5
    • Action.c(10): Start offset: 3, End offset: 5

    Start offset: 1 and End offset: match substring "bcdef".
    Note4: End offset is the first character after the end the current substring. That's why character "g" (with index 6) is not a part of matched string.

    As I've written in Note1, "bcdef" is the entire portion of string that was matched.
    Others items from an array relate to matched subpatterns.


    What is a subpattern in Regular Expression?
    It is a part of the RegExp pattern surrounded with parenthesis - "(" and ")".

    It's easy to get out the order of subpatterns. Just look through your pattern from left to right. When you find an open parenthes, this is a start of the current subpattern.
    Subpattern can be embedded.

    So, others captured subpatterns are:
    • Start offset: 2, End offset: 5 matches substring "cde".
      Note: current subpattern is "([acqz](.*))".
    • Start offset: 3, End offset: match substring "de".
      Note: current subpattern is "(.*)".

    As you can see - this is not so difficult. :)
    Regular Expressions can be very powerful and useful in LoadRunner.
Another example:

Let's practise with an example I mentioned early:
For example, we have to extract the name of month from these strings:
  • "abc 1st of May 2008xyz"
  • "abc 25th of February 2031"
  • etc
We can use the following pattern:
  • \d.+([A-Z]\w+)\s+\d{4}
The name of month will be matches by subpattern ([A-Z]\w+).

Please, see LoadRunner script, which captures and prints name of months:
Note: Pay attention that I use arr[1] to get info about substring.
As you remember, arr[0] contains info about the entire matched pattern, arr[1], arr[2], and so on contain info about matched subpattern.


Summary:
I've explained, shown and demonstrated how to use Regular Expressions (RegExp) in LoadRunner.
I hope, this knowledge will help you to create advanced LoadRunner scripts.

LoadRunner Correlation with web_reg_save_param_regexp

Do you have a correlation which you can’t solve because the values of the left and right boundary are dynamic? Correlation is an essential part of performance test scripting and there are plenty of different challenges with correlation. Imagine having a value of “GraphA123567EndGraphA” and the goal is to correlate 123567
From the example above, the left and right boundaries would be “LB=GraphA” “RB=EndGraphA”
What if the word GraphA is dynamic and can be anything from GraphA-GraphZ?
There is a solution at hand!
Using web_reg_save_param_regex will allow the user to grab a correlation value using dynamic left and right boundaries. This function uses the power of regular expressions, below are a few examples:
Example 1:
Source: “GraphA123567EndGraphA”
Solution: web_reg_save_param_regexp(“ParamName=CorrValue”, “RegExp=\“Graph[A-Za-z]\”, \“([0-9]+)\”, \“EndGraph[A-Za-z]\””, LAST);
Result: 123567
Example 2:
Correlate the values from a drop down list of a form
Source: dropdown >>> red, blue, green
Solution: web_reg_save_param_regexp(“ParamName=ColourList”, “RegExp=option=[0-9]+>([A-Za-z])
  • {ColourList1}=red
  • {ColourList2}=blue
  • {ColourList3}=green
Example 3:
Correlate up till the end of 642
Source:J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642″];
Solution: web_reg_save_param_regexp(“ParamName=SecurityString”,”RegExp=\”([A-Z0-9a-z\\\\+]+==\\|[0-9]+)\”\\];”,LAST);
Result:J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642
Example 4:
Correlate only “634998513832503642” Source:
<em>J\u002blsGd3zj1qdP\u002bvk0vDRaKyJFde5tCa6spDEy08SNab1hP8j5GTs4j6\u002f\u002bTqOwvxMHEQZLWd\u002btu8NlHJrVAarIQ==|634998513832503642"];</em>
Solution:
web_reg_save_param_regexp("ParamName=SecurityString",
    "RegExp=\"[A-Z0-9a-z\\\\+]+==\\|([0-9]+)\"\\];",
    LAST);
Result: 634998513832503642
So what is a Regular Expression? 
Also known as regex, a regular expression is a search string which enables matching of a string. Think of it as an advance searching function which can pick out values from a string of multiple characters.
Examples of regex:
  • \d matches a single digit
  • \w matches a single word (including alphanumeric characters and underscore)
  • [A-Z]+ matches any word which is upper case
  • [a-z]+ matches any word which is lower case
  • [0-9]+ matches any numeric value
There are other alternatives to web_reg_save_param_regexp. However these functions are limited and not as flexible.
LB/DIG RB/DIG – # will be a wildcard for a numeric value“session_id_##”
  • Adding LB/IC/DIG will ignore case
  • “LB/IC/DIG=session_id_##=” (e.g. Session_id_20)
LB/ALNUM or RB/ALNUM – ^ will be a wildcard for an alphanumeric value
  • ALNUMIC – ignore case
  • ALNUMLC – match only lower case
  • ALNUMUC – match only upper case
SaveOffSet
  • If there is a dynamic value for a boundary e.g. “session_id_2” (3,4,5)
  • SaveOffSet = 2 (to cover “2=”)
  • Web_reg_save_param(“SessionID”, “LB=session_id_”, “RB=\””, “SaveOffSet=2”, LAST);
LR implementation
  • PERL based
  • LR 11 does not support multiple capture groups however this is now supported in LR 11.52 (example below)
Example Multiple Capture Groups
Source: rows”:[[“NW,RO,RA”,”DLY”,”10/07/2011″,”10/17/2011″,”10/01/2011″,”RA”,”Y”,”FR”,”AMEA”,”AC”,”1945″,”50″,”50″,”AC 100IOSH-08″,”UserDefined”,”10/07/2011″,”Reassigned”…”
Solution: web_reg_save_param_regexp(“ParamName=ParamValue”,”RegExp=rows”:\[\[“[^”\r\n]*”,”([A-Z]{3})”,”[^”\r\n]*”,”[^”\r\n]*”,”[^\/]+\/[\d]+?\/2011″,”[A-Za-z]*”,”[^”\r\n]*”,”[^”\r\n]*”,”([^”\r\n]*)”,”[^”\r\n]*”,”([^”\r\n]*)”,LAST);
Result:
  • {ParamValue1} = DLY
  • {ParamValue2} = AMEA
  • {ParamValue3} = 1945

Recording Options – for web Vusers

Before recording a Web session, you can customize the recording options. 
This section includes a screenshot of the recording options available (LoadRunner 9.5 version)


First to start with
General Options

Under this you have three subsections 1. Script                     2. Protocols                 3. Recording

1. Script:  Here, you can specify a language for script generation with respect to the selected protocol for recording (Example: Web (HTML/ HTTP), Web (Click and Script) etc).

Script can be generated in any of the following programming languages

C, C#, Visual Basic, Visual Basic .NET, VB Script, and Javascript.
After selecting the language for recording you can select/ check particular recording options available. Forl Example, below screenshot elaborates the Scripting Options available for C Language

After recording you can use C Language pre-defined functions and control/ conditional statements to enhance the script as per the business requirement.

2. Protocols subsection lists the protocol(s) selected for recording.
Below screen shot refers to recording of Web (HTTP/ HTML) Protocol


3. Recording

Recording subsection lets you specify recording level that specifies what information and which function to generate while recording a Vuser script
The available levels are GUI-based script, HTML-based script, and URL-based script.

Note: GUI-based script is not available for web (HTTP/HTTP) protocol


HTML-based script, generates web functions for each user actions. The scripts are intuitive but java script emulation is not possible in this mode.
It is the default recording level for Web (HTTP/HTML) Vusers and it instructs VuGen to record HTML actions in the context of the current Web page. It does not record all resources during the recording session, but downloads them during replay (thus preferred recording level for HTML based scripts because of less correlation challenges comparatively).

Setting Advanced HTML-Based Options allows you to Specifying Script Types and Handling Non HTML-Generated Elements
For HTML-based scripts, you can specify the type of script:
·         A script describing user actions
·         A script containing explicit URLs only

A script describing user actions, is the default option and it generates functions that corresponds to Vuser’s action taken while recording.
For Example: When a Vuser clicks on a hyperlink then web_link is created. Similarly for submission of a form, a subsequent web_submit_form is created.

Unlike, a script containing explicit URLs only, where all the Vuser actions are recorded as web_url statements and web_submit_data (in the case of forms).
This option is quite useful when AUT (Application Under Test) has many hyperlinks, images, text fields etc in it’s webpage) and it’s used most of the times.

Handling Non HTML-Generated Elements
The following options are available:
·         Record within the current script step (default):
It is indicated with ‘EXTRARES’ flag. All resources are listed within web function as individual step itself under the EXTRARES.
·         Record in separate steps using concurrent groups
It enables VuGen to create a separate web function for each non-HTML elements and they are surrounded by web_concurrent_start ( ) and web_concurrent_end () functions.
·         Do not record
When you select this option, VuGen does not to record any of the resources generated by non-HTML elements.

URL-based script, generates web functions for each object on the web page. It automatically records every HTTP resource as URL steps (web_urlstatements of LoadRunner’s API). Thus the script will be lengthy (comparatively more number of LR functions are generated during recording because every object is recorded as corresponding web_url step in the Vuser script) and it is more prone to correlation too.
VuGen lets you set advanced options for the URL recording mode in the following area:
·         Resource Handling
Create concurrent groups for resources after their source HTML page option is enabled by default. Enabling this option allows VuGen to record URLs and its resources with concurrent groups.
·         Generating Custom HTTP Requests
By enabling this option you instruct VuGen to record all the resources (HTML and non-HTML), regardless of their content, as a separate web_custom_request statement.

HTTP Properties of Recording Options

HTTP Properties has two subsections:-
1. Advanced                           2. Correlation

Screenshot for reference

(i)   Under HTTP Properties > Advanced option, you can check/enable following preferences
·         Reset context for each action,
·         Save snapshot resources locally,
·         Generate web_reg_find function for page titles and its sub frames,
·         Add comment to script for HTTP errors while recording and Support charset

(ii)    It also allows you to Record script using earlier recording engine

(iii)   Moreover, it allows you to select Recording Schemes

HTML Mode Vs URL mode of recording in HTTP/HTML protocol

There are two recording modes available in LoadRunner to record the user actions of a Web Application. These are:
  • HTML mode
  • URL Mode

HTML mode – In this mode of recording, each USER ACTION is recorded as a separate request.  This is also called as Context based recording of the Web Application.
Assume a case where you are given a step by step document that has to be followed. The steps can be like launch the web Application  –> Enter the user name and password and click on Log in  –> Click “View Demo” link –> enter the personal details and click on submit –> View the Demo –> Log out.
An actual novice user would start browsing through the steps 1, 2 and stop if they didn’t find the link “view Demo”.  A actual user (human) can apply his intelligence and probably assume that “click here to get the demo” is same as “View Demo”, but a tool would not – causing the script to be terminated
This is the case case where the tool has to enter some data in a form and if the form is not found, the tool will halt the execution of the script.  Every request is directly dependent on the previous request’s response and has high chances of failure with the UI changes to the web application
The advantages of using the HTML recording mode is that the size is very compact and the customization efforts reduced.   However, with UI changes to the web applications the scripts would require much higher maintenance costs.
URL Mode: In this mode of recording each resource requested by the user is recorded as a separate request.  When a web site is launched apart from the HTML content, there will be images, java script files, CSS files an other resources downloaded extenze information. Each of these resources is recorded as a separate request.
URL mode is also called a context less recording because this mode does not depend on the UI of the application, rather the actions associated with the user actions performed on UI. As each resource is recorded the size of the script is large.  This means that a large amount of customization is also required. The benefit of having scripts with URL recording mode is that the maintenance cost associated with these are reduced and can be used across various releases of the product making the script less susceptible to changes in the UI .
URL mode of recording is usually used with Non –browser applications e.g. Any thick client activity is generally recorded using the URL mode.
The Trade off – with the HTML recording mode, another option is available under “Advanced options” of HTML mode  is Explicit URLs only mode.
The benefits of the HTML mode and URL mode are cobbled together in HTML  -  Explicit URLs only. With this mode, the size of the script is compact (as only the user action are recorded as requests, not at the UI level but at the request level).  This requires more customization effort, but has the advantage of high maintainability for longer durations. This is generally the most commonly used and recommended recording method for web applications where the scripts have to be maintained for longer durations
Tip: If  you have forgotten to record the script using HTML –> Explicit URLs mode? Then change the recording options to HTML –> Explicit URLs only, navigate to Tools –> Regenerate script. The regenerated script is as fresh as a recorded script using HTML Explicit URL’s only mode. Note:: Whatever any change made to the script previously will be lost when the script is regenerated.

Tips for handling recording problems in LoadRunner



When we were using LoadRunner for our Performance Testing engagements we have encountered certain recording problems. We have listed down some of the most common problems and the steps that should be followed to troubleshoot those issues.Problem 1: NOT PROXIED error in the recording events log.
This error occurs when there is some sort of spyware software installed on the system.
To troubleshoot this issue follow the below steps:
1. Use process explorer to get the list of dlls getting hooked into Internet Explorer.
2. Compare the list with the dlls from a machine where you can record.
3.Run Ad-Aware software. To download this, visit http://www.lavasoftusa.com/software/adaware/. This usually eliminates the spyware programs.
4. Make sure to check carefully the processes running on the machine. If you find any suspicious DLL\exe, just google the name and see if it’s a known spyware. If so, uninstall the software related to this DLL\executable.
5. Sort the list of DLLs by ‘Path Name’ in Process Explorer. The DLLs to scrutinize carefully are the ones which are not located in LoadRunner\bin and c:\windows\system32 or c:\winnt\system32.
Problem 2: “Cannot connect to remote server” in the recording log
This error occurs when the communication from this server and on that particular port should not be filtered
To troubleshoot this issue follow the below steps:
Try port mapping if it is a HTTPS site.
Go to Recording Options > Port Mapping and add the Target Server’s IP address, Port Number as 443, Service ID as HTTP, Connection Type as SSL. Make sure “Test SSL” returns success. Change the SSL Ciphers and SSL Version to achieve success for “Test SSL”.
Problem 3: “Connection prematurely aborted by the client” error in the recording log.
This error occurs when the communication from this server and on that particular port should not be filtered
To troubleshoot this issue follow the below steps:
Try port mapping with Direct Trapping. Here’s how we can do direct trapping
a. Enable the Direct trapping option:
[HKEY_CURRENT_USER\Software\Mercury Interactive\LoadRunner\Protocols\WPlus\Analyzer]
“EnableWSPDebugging”=dword:00000001
“ShowWSPDirectTrapOption”=dword:00000001
“EnableWSPDirectTrapping”=dword:00000001
b. Before Recording Multi-Web, add port mapping entries for HTTP/TCP (and NOT SSL) connections.
Set their ‘Record Type’ from ‘Proxy’ to ‘Direct’.
c. Start recording.
Problem 4: Application sends a “Client Side” certificate to the server and waits for server authentication.
This error occurs because LoadRunner sits in between the client and server, it takes this certificate information from the client and pass it on to the server. So, you need to have the certificate in .pem format.
To troubleshoot this issue follow the below steps:
Use openSSL to convert the client side certificate to .pem format and add it in the port mapping in recording options
In Internet Explorer:
1. Choose Tools > Internet Options. Select the Content tab and click Certificates.
2. Select a certificate from the list and click Export.
3. Click Next several times until you are prompted for a password.
4. Enter a password and click Next.
5. Enter a file name and click Next.
6. Click Finish
In Netscape:
1. Choose Communicator > Tools > Security Info.
2. Click on the Yours link in the Certificates category in the left pane.
3. Select a certificate from the list in the right pane, and click Export
4. Enter a password and click OK.
5. Enter a filename and save the information.
The resulting certificate file is in PKCS12 format. To convert the file to PEM format, use the openssl.exe utility located in the LoadRunner bin directory. To run the utility:
Open a DOS command window.
Set the current directory to the LoadRunner bin directory.
Type openssl pkcs12 -in <input_file> -out <output file.pem>
Enter the password you chose in the export process.
Enter a new password (it can be the same as before).
Enter the password again for verification.
In Recording Options > Port Mapping > check the option for “Use specified client side certificate” and point to the saved .pem file.
Problem 5: Server sends a certificate to the client for authorization and only when authorized by the client a connection can be established between the client and server. This is mostly true for Java based applications. The error we get is “No trusted certificate found”
This error occurs since the recorder is in between client and server, you should authorize the certificate from the server. So, your wplusca.crt should be in the server’s certificate repository.
To troubleshoot this issue follow the below steps:
Copy wplusca.crt file into the certificate repository of the server.
1. Locate keytool.exe executable (usually under JRE_ROOT\bin directory).
2. Add its directory to the PATH environment variable.
3. Locate cacerts file (with no extension) under JRE_ROOT\lib\security directory.
4. Copy the attached cacert.pem to this directory.
5. Make this to be the current directory in the command prompt.
6. Execute:
keytool -list -keystore cacerts
7. It will ask you for the password, reply:
“changeit”
8. It will list all the CA certificates installed (usually 25).
Notice the number of certificates.
9. Execute:
keytool -import -file cacert.pem -keystore cacerts -alias mercuryTrickyCA
10. It will ask you for the password, reply:
“changeit”
11. It will ask you if you are sure you want to add the certificate to the store, reply:
“yes”
12. Execute:
keytool -list -keystore cacerts
13. It will ask you for the password, reply:
“changeit”
14. It will list all the CA certificates installed.
The number of certificates should be bigger by 1
Problem 6: The recording log does not indicate any error but the connection between the client & Server has ended abruptly. This could happen possibly due to the ‘WPLUSCA’ certificate with a private key
This error occurs when because the recorder sends the ‘WPLUSCA’ certificate with a private key to the server for authentication while recording.
To troubleshoot this issue follow the below steps:
1. Go to LoadRunner\bin\certs and make a copy of the WPlusCA.crt file.
2. Open it in a notepad
3. Delete the portion between “Begin RSA Private Key” and “End RSA Private Key”
4. Save the file.
5. Now right click the file and choose “Install certificate” and follow the on screen instructions.
This will now send a certificate without the private key and if the server accepts the communication will continue.
Problem 7: “Failed to resolve hostname” error in the recording log
If VuGen box is not able to resolve the server name that is given in URL, it throws this error in the recording log and then crashes the IE
To troubleshoot this issue follow the below steps:
Modify hosts file on the vugen machine with server name and any IP address
The above problems are some of the common problems when we are using LoadRunner for Performance Testing. The above mentioned troubleshooting steps will help you to fix these issues if they occur during your performance testing assignment.