Kit name: VSI-ALPVMS-T4.ZIPEXE Installing T4 Version V4.4D 1 Sep 2012 16 Apr 2015 -- Minor edits for VSI release Dec 2016 -- A few more minor edits for VSI release Feb 2017 -- Add problems fixed in release V4.4D WELCOME TO VSI T4 VERSION 4.4-D Thank you for using T4. This unsupported kit is supplied for VSI Alpha OpenVMS V8.4-2L1 or later. To get maximum benefit from the T4 data you will be able to collect with this kit, you will want to download the TLViz and CSVPNG kits which are available from the same web site. TLViz and CSVPNG are specifically designed to save you time when you are analyzing the CSV (Comma Separated Value) files that the T4 collector creates. If you have questions about T4, TLViz, CSVPNG or other Friends of T4, or if you would like to be on the mailing list, please send mail to T4@vmssoftware.com. This document contains the following sections to help you get started with T4 INSTALLING THE KIT THE T4 V4.4 KIT PRIVILEGES REQUIRED FOR T4 INSTALLATION RESET FILE ATTRIBUTES DEASSIGN T4$SYS LOGICAL NAME INSTALL THE LATEST T4 KIT MAKING THE T4 LOGICAL NAMES PERMANENT CREATING A T4$DATA STORAGE AREA SET UP A LOCAL BATCH QUEUE TO RUN T4 RUNNING T4 QUOTAS REQUIRED TO RUN THE T4 COLLECTION PRIVILEGES NEEDED TO RUN T4 COLLECTION DISK SPACE ESTIMATING DISK SPACE REQUIREMENTS RUNNING A T4 V4.4 TRIAL SESSION LAUNCHING T4 IN LONG-TERM HISTORY MODE TYPES OF FILES CREATED IN T4$DATA T4 V4.4D RELEASE NOTES T4 V4.4 RELEASE NOTES KNOWN ISSUES WITH T4 V4.4 TERMS & CONDITIONS FOR USE OF T4 V4.4 INSTALLING THE KIT THE T4 V4.4 KIT Your zipped up T4 kit includes the following files: 1) VSI-VMS-T4-T0404-D-1.PCSI$COMPRESSED - The T4 V4.4 PCSI kit. 2) T4_README.TXT - The text file version of this document. 3) VSI-VMS-T4-T0404--1.PCSI$COMPRESSED_VHF - CDSA encrypted, signed manifest PRIVILEGES REQUIRED FOR T4 INSTALLATION You will need the SYSNAM privilege to establish the logical names required to successfully install T4 and make it ready to run. RESET FILE ATTRIBUTES Once you have moved this PCSI kit to your OpenVMS system, you may need to RESET the FILE ATTRIBUTES. These can sometimes be altered in transit depending on how the transfer was accomplished. If necessary, the appropriate reset can be accomplished with the following command: $ set file/att=(rfm:fix,lrl:512,mrs:512,org:seq,rat:none) VSI-VMS-T4-V0404-D-1.PCSI$COMPRESSED DEASSIGN T4$SYS LOGICAL NAME If you are already running a version of T4, be sure you DEASSIGN your current T4$SYS system wide logical name before installing the new kit. $ DEASSIGN T4$SYS /SYS INSTALL THE LATEST T4 KIT You can now install the latest T4 V4.4 kit using: $ PRODUCT INSTALL T4 This installs the kit to the [.T4$SYS] subdirectory beneath SYS$SYSDEVICE:[VMS$COMMON] (that is, SYS$SYSDEVICE:[VMS$COMMON.T4$SYS]) To install to a different directory, use the /DESTINATION option as follows: $ PRODUCT INSTALL T4/DESTINATION=disk$somewhere:[dir.tree] This installs the kit to the [.T4$SYS] subdirectory beneath the destination directory (that is, disk$somewhere:[dir.tree.T4$SYS]) The T4 kit must be installed separately on each node where you expect to run it. Once you have installed the T4 V4.4 kit, you will find many more files of additional reference material. MAKING THE T4 LOGICAL NAMES PERMANENT This is a very important step that helps make sure you get full use out of T4 on your systems. By making key T4-related logical names permanent, you enable interrupted T4 sessions to restart following a system reboot and make sure that you do not miss any vital data. The logical names of interest are T4$SYS and T4$DATA, which are discussed in the following sections. Define the T4$SYS logical and make it permanent by adding the definition to the system startup SYLOGICALS.COM command procedure. For example, try the following command. $ DEFINE /SYSTEM /EXEC T4$SYS SYS$SYSDEVICE:[VMS$COMMON.T4$SYS] CREATING A T4$DATA STORAGE AREA When you run T4 collections you will need a convenient and suitably sized disk location to store the T4 generated performance data. Set up your data disk in advance and point to it with the T4$DATA logical name. It is equally important to make this logical name permanent by adding the definition to the system startup code SYLOGICALS.COM command procedure which allows T4 sessions to resume following system reboots. $ DEFINE /SYSTEM /EXEC T4$DATA Your_Data_Disk:[000000.T4$DATA] You can establish a separate T4$DATA area for each OpenVMS system or you can set up a single T4$DATA disk that is available to all nodes on your OpenVMS cluster. WARNING: Do not use your system disk for T4$DATA. SET UP A LOCAL BATCH QUEUE TO RUN T4 For each OpenVMS node that you will want to monitor with T4, you will need to use a local batch queue to run the T4 jobs on that node. This can be an existing queue (assuming it has available job slots) or you can set up a special T4 batch queue for that node as shown below. First, start the queue manager if not already started $ start/queue/manager/new Create and start up a new batch queue $ init/que/start/batch T4$batch /job=10 $ sho queue/batch/full Batch queue T4$BATCH, idle, on NODEX:: /BASE_PRIORITY=4 /JOB_LIMIT=10 /OWNER=[SYSTEM] /PROTECTION=(S:M,O:D,G:R,W:S) You will need one batch queue and one T4 collection session for each OpenVMS node you are interested in. As noted above, all the collected and processed data from these sessions will be saved to the T4$DATA directory. RUNNING T4 QUOTAS REQUIRED TO RUN T4 COLLECTION The User Account (e.g. T4_USER) that you plan to use to run T4 will require the following QUOTAS. PRCLM must be at least 20. TQELM must be at least 100. PGFLQUOTA must be at least 500000. $ mc authorize UAF> mod T4_USER /prclm=20/tqelm=100/pgflquota=500000 UAF> exit PRIVILEGES NEEDED TO RUN T4 COLLECTION For the user who will be launching T4 history creation sessions, the ALTPRI privilege is needed so that the OpenVMS Monitor Utility can run at the recommended process priority of 15. DISK SPACE ESTIMATING DISK SPACE REQUIREMENTS For your most important and performance sensitive systems, we suggest that you launch T4 in long-term history mode by answering yes when asked the question: Re-Submit data collection job daily [N] : Y This way it will build up a detailed performance history for you day by day. The actual size you will need for your T4 history area depends on several key factors: * the number of nodes under measurement, * the number of hours of measurement each day, * the number of devices to be measured * the number of processes on each system, * the sampling rate (default = 60 seconds) * the OpenVMS version To gauge the amount of disk space that you will require, we suggest you start with at least 500,000 blocks. Run a trial session (see quick instructions below) of 1 to 2 hours and determine how much disk space is needed for that run. Then adjust the size of the T4$DATA storage area as necessary to meet your needs. WARNING: Do not use your system disk for T4$DATA T4 includes some rudimentary capabilities for assisting you in the management of your T4$DATA performance history area. Since the data that T4 helps you collect may prove invaluable to you in the future, you will want to think through and apply your standard local site policies for backing up, archiving, and preserving this potentially priceless historical system information. RUNNING A T4 V4.4 TRIAL SESSION Once you have installed T4, set up your T4 user account with the appropriate privileges and process limits, established a T4 Batch queue on each node you will be monitoring, and created a T4$DATA area, you are now ready to launch your first T4 V4.4 collection session. We suggest you run a trial session to help calculate the disk space that you will require to run T4 in its recommended long term history mode (see the OpenVMS Technical Journal Article at http://h71000.www7.hp.com/openvms/journal/v3/t4.pdf for a full explanation of the benefits of long term performance histories To help estimate the amount of disk space you will require, run the following command and respond to the questions you are asked following the guidelines shown below. $ @T4$SYS:T4$CONFIG.COM Start Time -- pick a time that is 5 minutes in the future as this will give you enough time to work your way through all the questions. That way the T4 collection session will be launched prior to the time you specify and you can make sure that you get a full hour of data. End Time -- pick a time that will give you a total run time of one hour. Batch queue -- remember to use your T4$BATCH queue that is local to the node you are measuring Network Interface Device -- Enter a question mark to get a list of all available network interface devices. Then, highlight that list, and paste it in as the answer to the question. This will cause T4 to monitor each and every one of your Network adapters. Note that you can also provide the string ALL in order to monitor all your network adapters. Sampling Interval -- use the default 60 seconds Data Directory -- use T4$DATA Re-submit data collection job daily -- answer NO to this question as you are using this run to help you determine storage needs before launching T4 in long term history collection mode. Email Address -- if you can send email from the VMS node being measured, fill this in so it sends email notifying you that the data has all been collected and processed. Once the run is complete, check the sizes of the files created in the T4$DATA directory to discover the approximate storage costs per hour for this OpenVMS node. See the sections below describing the types of files created during a T4 session and the suggested retention periods of each type of data. LAUNCHING T4 IN LONG TERM HISTORY MODE Having established a reasonable size for your T4$DATA area by following the steps outlined above, you are now ready to launch T4 in long term history mode. Long term history mode for T4 is explicitly designed to help you maximize the benefits that T4 can provide. At the same time it helps to minimize how much time you will have to spend and keep that down to an absolute bare minimum. Here are some recommendations for parameter values to use in response to the questions triggered by running the following command $ @T4$SYS:T4$CONFIG.COM Start Time -- use the default which will be tomorrow at one minute after midnight. End Time -- use the default which will be tomorrow at 23:59. These settings for Start Time and End Time mean that you will have round the clock performance data for the systems that are most vital to you. Batch queue -- remember to use your T4$BATCH queue that you created and that is local to the node you are measuring. Network Interface Device -- type in the names of the devices that are a regular part of your production environment. You can enter a question mark to get a list of all available network interface devices. If you want to measure all of them, you can highlight that list, and paste it in as the answer to this question. Sampling Interval -- use the default 60 seconds. This has proved to be an excellent default compromise value for long term history creation. Data Directory -- use T4$DATA Re-submit data collection job daily -- answer YES to this question. When you do this, each time T4 runs, the first thing it will do is submit a new batch job for the following day. If you have established T4$DATA and T4$SYS as system wide logical names in your SYLOGICALS.COM file, then this single launch operation will create a full long term history for this node. Email Address -- using this or not using this feature is your call. Your T4 collections will start tomorrow at one minute after midnight and continue collecting and saving vital performance data for you until the batch jobs are deleted. Welcome to the world of T4. We look forward to hearing of your experiences in using these handy tools. TYPES OF FILES CREATED IN T4$DATA T4 produces a composite CSV file daily for each node being monitored. The names of these files are of the form: T4_____COMP.CSV For example, a one hour run on node PRFE40 might look like: T4_PRFE40_28JUN2005_1400_1500_COMP.CSV These *COMP.CSV files are normally the first thing that we look at. The output of a T4 collection session includes CSV files ZIP files LOG files DAT files (for OpenVMS MONITOR and for T4FCMON) Please note that, after the CSV and DAT files are ZIPped, T4 will delete them, leaving only the ZIP files. The key CSV files are: *COMP.CSV *DISK.CSV *SCS.CSV *T4FCMON.CSV The remaining CSV files are what we refer to as intermediate files and they can be deleted after a few days. We recommend that you retain all the ZIP files which contain CSV's for at least eighteen months. We recommend retaining the ZIP files which contain DAT's for at least 45 days. Even beyond eighteen months, if you find you must remove the older CSV files, we suggest you send these to a permanent archival storage location that you can access as needed. You'll never know when you will have a need to search back 2 or 3 years to compare what is happening right now to how things used to be. You will only be able to do this if you carefully guard all of these invaluable performance timeline files. T4 V4.4D RELEASE NOTES 1. Multiple Bug fixes a. In opposite to the other collectors, the memory collector created a CSV file that was not accessible until the collection was finished. That has been changed, now the MEM CSV file is also sharable. b. The displayed version number in T4$CONFIG.COM was 4.3. This has been corrected to 4.4. Also the displayed copyright has been adjusted. c. The process name that was created in T4$COLLECT.COM contained the number 43. Corrected to 44. d. There was an error in WBMSDA.COM. At one place the variable collsecs instead of collsecs_time was used. That has been corrected. e. In T4$CONFIG.COM there are the triggers for PEDriver collection, memory collection and shadowing collection. The question to collect the shadow server information was never asked. That has been corrected. f. WBMSDA.COM and HBMMDATA.COM are creating CSV files in the T4$DATA directory. The user is asked during running T4$CONFIG whether T4$DATA should be the working directory. If the user is not taking the default then the CSV files created by the mentioned two DCL scripts are going to a different directory than all other CSV files. This has been corrected, now T4$CONFIG.COM is passing the working directory name to WBMSDA.COM and HBMMDATA.COM. g. T4$FC_MON.COM called different executables. If OpenVMS was 8.2-1 or older then T4$Fc_Monitor_Old.Exe was executed, otherwise T4$Fc_Monitor_New.Exe. That makes no sense for a VSI product where OpenVMS is always 8.4-1H1 or later. The reference to the old executable has been removed and the executable T4$Fc_Monitor_New.Exe has been renamed to T4$Fc_Monitor.Exe. T4LNK.COM creates only T4$Fc_Monitor.Exe. h. Checks for V8.2 has been disabled in T4$COLLECT.COM. They make no sense anymore, and they could cause T4 to fail if a fieldtest version is used. i. The image list in T4$COLLECT.COM and T4$VERSION.COM has been modified to be up-to-date. j. The output of T4$VERSION and the log file of T4$COLLECT are showing an error on CSVPNG-ALPHA.EXE which is reproducible outside T4. Analyze/Image/NoOutput/Select=(Identification=Image,Link_Time) t4$sys:csvpng_alpha.exe is returning %ANALYZE-E-ILLFIL, Illegal file format encountered. This is an HP image. This issue has no harm. To avoid confusion the image name has been dropped from the list of images to be analyzed. T4 V4.4 RELEASE NOTES T4 V4.4 contains a number of changes to the previous release (T4 V4.3-1), as follows: 1) Added new collector for PEDriver Virtual Circuit (VC) charactersticsfor each node in the cluster. If the system is a cluster member, when the user runs T4$CONFIG.COM, he is asked the following question : Collect PEDriver VC statistics? [N] : Answer with 'Y' to collect PEDriver characteristics. The following characteristics are collected by the PEDriver collector : Bytes Xmt Cumulative size of all the messages transmitted over the VC, measured in bytes. Bytes Rcv Cumulative size of all the messages received over the VC, measured in bytes. Buffer Size Maximum data buffer size for this VC Msg Xmt Indicates the number of transmits completed along this VC. Msg Rcv Indicates the number of messages received over this VC. Buf Size Incr Number of times buffer size has increased for this VC. Msg_Xmt_Unsequence Number of unsequence messages transmitted Msg_Rcv_Unsequence Number of unsequence messages received Load_Class Bandwidth of the VC. Msg_Xmt_Sequence Number of sequence messages transmitted Msg_Rcv_Sequence Number of sequence messages received Open_Channels Number of currently open channels available to the virtual circuit. Seq_TMO_Ratio Ratio of number of sequenced messages transmitted to number of retransmits Msg_Rcv_Cached Number of messages in the receive cache TR_Rcv_Cache_Size Maximum size of the TR receive cache No_ECS Number of times VC got closed due to no usable channels (No ECS) No_Xmt_Path Counts the number of times channnels not available for transmit Round_Trip Average round-trip time, in microseconds, for a packet to be sent and acknowledged SeqMsg_RetryExh Number of times VC was closed due to sequence message transmission exhaustion for a packet. Rcv_Cache_Miss Number of messages that could not be placed in the cache because message sequence number difference was out of cache's range Round_Trip_Variance Average deviation, in microseconds, of the round-trip time. DL_Xmt_Incmpl Account for VC close due to transmit completion wait exhaustion. Illegal_Seq_Msg Number of illegal sequence messages received Xmt_Window_Full Number of times transmit window reached its maximum. Buf_Size_Decr Number of times ECS buffer size got decreased resulting in VC closure Bad_Checksum The number of messages received with bad checksum. CmdQLen_Cur Reflects the number of Port commands seen in the queue currently. CmdQLen_Max The maximum value reached by this field. CC_DFQ_Empty This reflects the number of times the DFREEQ became empty Rcv_Short_Msg Number of received messages too short for NISCA. Rmt_TR_Rcv_Cache Receive cache size of remote node at TR layer NPAGEDYN_Low Number of VC closures due to unavailability of non--paged pool. TR_MFQ_Empty Number of times the MFREEQ became empty Cur_UnAcked_Msgs Current number of outstanding unacknowledged messages XmtWin_Cur Current transmit window size. XmtWin_Slo Slow growth threshold for the transmit window XmtWin_Max Maximum transmit widnow size Grow_Cnt Number of succesful transmits since transmit window size was increased Grow_Thresh Number of succesful transmits required to increase the size of the transmit window. Member_Count Number of ECS members Remote_BUSes Number of unique remote buses across the ECS. Local_BUSes Number of unique local buses across the ECS. Demote Current threshold for reclassifying a FAST channel to SLOW Epochs Number of times all ECS membership criteria was recalculated Promote Current threshold for reclassifying a SLOW channel to FAST. New_ECS Indicates the number of times the ECS was formed by selecting all active channels based on previously established ECS membership criteria. Faster Current threshold for reclassifying a channel as FASTER than the current set of ECS channels. Min_Rmt_DL_Buf Indicates the minimum size of the communicating node�s (remote) datalink receive ring buffer. 2) Added new collector for Shadowing characterstics. If SHADOW_SERVER process is present, when the user runs T4$CONFIG.COM, he is asked the following question : Collect Shadowing statistics? [N] : Answer with 'Y' to collect Shadowing characteristics. The following characteristics are collected by the Shadowing collector : Resets The number of bitmap resets for a shadowset during the collection interval. Note that by default bitmaps are only checked and if needed reset every 150 seconds (SYSGEN SHADOW_HBMM_RTC). So, collection intervals shorter than RTC should not have reset values > 1. % of Threshold The number of blocks represented in a DSA unit's bitmap at the end of a collection interval expressed as a percentage of its reset_threshold. SF_MSG_SENT (from WBM_DATA$GL_SF_MSG_SENT) The number of SCS messages sent in single message mode plus the number of full buffered message mode SCS messages sent during a collection interval from this system. TQE_MSG_SENT (from WBM_DATA$GL_TQE_MSG_SENT) is the number of SCS messages sent via TQE while in buffered message mode during a collection interval. When in buffered message mode a TQE is used to check buffers for connections in buffered message mode every WBM_MSG_INT (SYSGEN parameter). Buffered message mode status: For each SCS connection the buffered message mode status bit is save at the end of each collection interval. 1 indicates buffered message mode. Since a connection can switch into or out of buffered message mode as often as every WBM_MSG_INT (10msec by default) this information may not always be a good representation. 3) Added new collector for Memory characteristics. When the user runs T4$CONFIG.COM, he is asked the following question : Collect Memory statistics? [N] : Answer with 'Y' to collect Memory characteristics. The following characteristics are collected by the Memory collector : Pages Total The total number of pages in Main Memory. Pages Free The number of free pages in Main Memory. Pages Inuse The number of pages that are in use in Main Memory. Pages Modified The number of pages that have been modified in Main Memory. NPaged Total The total memory (in Bytes) in Non-Paged Pool. NPaged Free The free memory (in Bytes) in Non-Paged Pool. NPaged Inuse The in-use memory (in Bytes) in Non-Paged Pool. NPaged Largest The largest memory (in Bytes) in Non-Paged Pool. Paged Total The total memory (in Bytes) in Paged Pool. Paged Free The free memory (in Bytes) in Paged Pool. Paged Inuse The in-use memory (in Bytes) in use in Paged Pool. Paged Largest The largest memory (in Bytes) in Paged Pool. Gblpage Contig The maximum number of free, contiguous global CPU-specific pages. Gblpage Free The current number of free global pages. Gblsect Free The current number of free global section table entries. Pagefile Free The number of free pages in the currently installed page files. Pagefile Size The number of pages in the currently installed page files. Swapfile Free The number of free pages in the currently installed swapping files. Swapfile Size The number of pages in the currently installed swapping files. 4) Introduced a new logical "T4$PROCESS". This logical can be used instead of editing the "Process_Name_List" symbol in T4$COLLECT.COM to collect information about processes. The users can define the logical to a comma separated list of process names( wildcards can also be used like in the definition of the symbol "Process_name_list" currently ). Ex. $ define/sys/exec T4$PROCESS "*SHAD*,*T4*,*BATCH*" 5) Fixed the following issues : 1) Remove the trailing spaces in Time format. 2) Correct syntax error with SUBMIT command. 3) Move the symbol assignemt of "Mon_Tdc" above the $RESTART test. 4) Correct the symbol definition of "This_Proc" to work with all valid file specifications. 5) Fix the "%DCL-W-TKNOVF" error with T4 Version. KNOWN ISSUES WITH T4 V4.4: 1) PEDRIVER collector T4$PE_VC collects the data from SDA utility and parses the output to generate CSV file. In SDA when there is an overflow in the display, value is replaced by **** which affects the collector output. In such cases, the generated CSV files will not be usable with TLViz TERMS & CONDITIONS FOR USING T4 V4.4: T4 V4.4 is provided to you, subject to the following terms and conditions. Any comments or questions regarding T4 V4.4 or its terms and conditions can be directed to T4@vmssoftware.com. (a) T4 V4.4 is supplied 'as is.' without warranties, either expressed or implied. (c) T4 V4.4 is not a Commercial 'Off-The-Shelf' software product. (d) T4 V4.4 is neither freeware nor shareware, and cannot be freely distributed other than by VMS Software, Inc, subsidiaries, successors and assignees. (e) T4 V4.4 remains the sole intellectual property of Hewlett-Packard Enterprise Company and VMS Software, Inc., subsidiaries, successors and assignees. (f) T4 V4.4 may not be redistributed or supplied to any third party, either for commercial gain or otherwise without the prior written consent of VMS Software, Inc. (g) T4 V4.4 is deemed to be supported *only* by consulting services purchased directly from VMS Software, Inc, subsidiaries, successors and assignees, specific for this purpose and by prior agreement. (h) The recipient must not request any kind of support from any other VMS Software, Inc. entity, such as any Customer Support Center or Engineering group. This support restriction also applies to any issues of coexistence or interoperability with any other software or hardware, including (but not limited to) supported VSI or HP products. WHERE TO GET MORE INFORMATION If you have questions, suggestions, comments, or if you want to be on the T4 & Friends mailing list, then please send mail to both T4@vmssoftware.com.