mBrace Performance Modelling - Data Gathering

System Monitors

The transaction profiles are established by the server resource metrics and transaction response times obtained in the measurement environment, namely the test environment. The following paragraphs contain descriptions of collection methods which can be used to collect server resource utilisation metrics (CPU usage, disk I/O, memory, etc.) The production environment is often shared by multiple applications. It is therefore important to measure the load of the production environment over a long time period of at least a few months, in time intervals of about 5 minutes. The file format in which log files should be presented to the parsing engineer is described below: YYYYMMDD-[application]-[measurement]-[machine name=""]-[su mu="" bo=""].[measuring-tool]
The following measurement tools are readily available:
• Nmon
• Perfmon
• Sar
• Iostat
• Ps
• sbus
After each measurement, the collected measurement data should be compressed into a .zip file and sent to the mBrace Performance Analyst as soon as possible. Following delivery, measurement collectors are requested not to delete measurement data located on the respective application server platforms unless specifically requested by the mBrace Performance Analyst.

IBM - AIX

The Single-User and Multi-User transaction executions will require measurements to be captured on the AIX servers, at 1 second intervals (smaller intervals are not possible with Solaris). The measurement collection is started prior to the execution of any transactions and stopped when all transaction executions have completed, in order that “nMon” output files are generated.
The commands for outputting such files are: nmon -F [nmon-logfile] -r [machine name=""] -I 0.001 -t -s 1 -c
[counter] Layout of options:
-F outputfile, Path and filename
-r Machine name
-I Top process information.
-t Log top data -s time between snapshots in seconds. 1 = 1 sec, 300 = 5 min
-c snapshot counter Versions of NMON older than v12e cannot handle SMT configurations.

LPAR

It is very important for reliable measurement results that the following points are observed:
• Switch SMT off
• Dedicated LPAR configurations only (in other words, NO shared LPARs) or Capped LPAR where online Virtual CPU’s are equal to Entitled capacity (therefore equal to Online Virtual CPUs) Capacity Increment: 1

SMT, or Symmetric Multi Threading is a CPU option which enables 2 hardware threads to be executed in parallel on a single CPU. A single CPU therefore appears to the operating system as 2 seperate CPUs. However, this does not equate to the application being able toperform twice as fast, as would happen with a dual core CPU. Whilst SMT has the potential to deliver performance benefits of up to 30%, such benefits are largely dependent on taking advantage of multi-threaded application capabilities. When running the Multi-user measurements, the multiple user loads cause multiple processes and threads to become active, leading to misleading processor measurement results if SMT is switched on.
It is therefore necessary to switch SMT off.

How do I enable or disable SMT?
You can enable or disable SMT by running the smtctl command.
The following is the syntax: smtctl [ -m off | on [ -w boot | now]]
The following options are available:

-m off Sets SMT mode to disabled.
-m on Sets SMT mode to enabled.
-w boot Makes the SMT mode change effective on next and subsequent reboots if you run the bosboot command before the next system reboot.
-w now Makes the SMT mode change immediately but will not persist across reboot.
If neither the -w boot or the -w now options are specified, then the mode change is made immediately. It persists across subsequent reboots if you run the bosboot command before the next system reboot. Shared LPARs provide a distorted perspective of CPU utilisation.

Shared LPARs can access CPU cycles from a shared CPU pool if they require further CPU resources. However, this behaviour is not constant, but can fluctuate if the active LPAR is not eligible for spare CPU cycles, due to another LPAR on the same system having a higher priority. Since there is no fixed uppeer limit of available CPU cycles, this leads to distorted CPU utilisation metrics. The reported values for CPU utilisation in NMON are unreliable and cannot be used for mBrace model purposes.
For an explanation, please see the following post by Nagger, the author of NMON:
http://www-128.ibm.com/developerworks/forums/thread.jspa?messageID=14085487�

If it is not possible to measure on a dedicated LPAR, then the utilisation of the number of virtual CPUs allocated to the LPAR configuration, provide a distorted perspective of CPU utilisation. The CPU utilisation with regards to virtual CPUs is only available in the newest versions of NMON and the NMON logs can be found under the header named LPAR.

Network TCPDump

In a number of cases, it is usually useful to obtain network resource measurements along with the server resource utilisation measurements. The network trace data is collected when the transactions are being executed and this can be done with the use of the standard native UNIX tool, TCPDUMP. As the Tcpdump output file will contain all the network traffic information that occured at the monitored machine, ensure that there is sufficient space on the application server being monitored to contain the resultant tcpdump file. Note that the tcpdump output file can easily grow to 1GB per hour, depending on the application, so ensure that there is sufficient space.

UNIX Command: tcpdump –w [tcpdump-filename]

[tcpdump-filename]Note that the tcpdump utility should ideally be executed on each individual server. If this is not possible, ensure that the entire network traffic resulting from the transaction execution, is captured.
It is also helpful to ensure that any application server connection pool to the database is switched off.

Sun / Solaris

The Single and Multi-User transaction executions will require measurements to be captured on the Solaris servers, at 1 second intervals (smaller intervals are not possible with Solaris).
The measurement collection is started prior to the execution of any transactions and stopped when all transaction executions have completed, in order that “sar” or “vmstat” output files are generated. Note that the “Nmon” utility does not run on Solaris operating systems.

A vmstat and sar output files can be created using a shell script. The commands for outputting such files will be added later

Windows (Server and Client)

Set Perfmon Counters Please note that the following procedures may vary slightly depending on the Windows version being used. Set the Perfmon counters up using the procedure detailed below:
• Select “Start” on the Windows desktop, and thereafter select the “Execute” or the “Run” command, depending on the Windows version.
• Enter “perfmon.msc” in the entry box and click “OK” Make a new “Data Collector Set” , for 1 sec interval measurements and give it an appropriate name. Set the log output format to “Text File (Comma Seperated values)”. Set the Sample interval to 1 second.
Add the following performance counters to the Data Collector Set:
\PhysicalDisk(_Total)
\Disk Transfers/sec
\Processor(_Total)\% Processor Time
\Process(*)\% Processor Time \Process(*)
\ID Process
The measurement capture time period can be set under the tab page marked “scheme”.

VMWare

It is important to ensure that SMT is switched off.

Network (Network Monitor)

A network trace can be captured on Windows servers using Network Monitor 3.0 or Wireshark. Network traces can also be captured between application clients and web servers through the use of a network hub and a capture PC running in promiscuous mode. An effective capture requires that the capture PC and the clients all be connected to the hub. Whilst network trace captures of switches is possible, it is subject to burst activity and therefore has the potential to lose packets. Network Monitor 3.0 can be downloaded from the Microsoft Site, along with relevant installation and configuration information.

Oracle Database

With the help of the Oracle Trace function, metrics are collected for each execution of the Single User transaction types, over the time period when the Application server communicates with the Database server.
The metrics include the number of round trips made during such communication. In addition, data is needed with regard to the Oracle SGA memory utilisation incurred during the execution of the transactions.

Oracle Session Trace There are numerous ways to enable oracle tracing. Most commonly used is altering the session itself:
• Alter session set timed_statistics=true
• Alter session set max_dump_file_size=unlimited
• Alter session set tracefile_identifier=’[name]’ events='10046 trace name context forever, level 12' This can be done with a logon trigger when required.

It is also possible to enable tracing database wide. Please refer to the Oracle Database Performance Tuning Guide and Reference (The Java server should be stopped and started to ensure that a new trace file is created for every transaction type or transaction type group execution.)

Service Bus

The measurement functions in the adapters are used to determine the latency of the service calls. Before measurement metrics are retrieved, the various metrics counters must be reset. Metrics counters should be read, and reset after every meaurement.

Please follow the link to find out more about Testing Performance's Performance Modelling Services