Contents
Testmate Setup
The setup chapter describes the required services and apps for Testmate to operate
Firewall Rules
Required Services
To setup Testmate, Log in to Yarnman, open Administration App->Services
Testmate will require three services described below
Testmate App
Interconnect Service x 2
Terminator Service
From Top hand corner on services screen, select Testmate App from drop down
Define Service Name, select Yarnman Node and Authentication policy from drop downs and optionally Customer to be associated → Submit
Select Services Menu → Select Proxy Service
From Proxy Configuration Page, select ‘Service Routing’ from top right corner
Select ‘Add Yarnapp' and from drop down select the Testmate App ->Submit
The Testmate App has now been added and may be selected from top Right Drop down
Testmate requires 2 x Interconnect services. By default, there will always be one interconnect service running in Yarnman. To create a second service
Go Services → ‘Add Service’, select 'Interconnect Service
Name service, select Node/Arm, then Submit
Terminator service is created from Services → ‘Add Service’
Select Node/Arm and Submit
JTAPI nodes
Any node that runs JTAPI requires downloading the jtapi.jar from CUCM – this is required for Testmate
Browse to administration application and select the required node
Download and restart jade berlin service – it will ask what CUCM IP to download from. Enter the IP and OK
CUCM Configuration
Yarnman CUCM app user requires
Standard CCM End Users
Standard CTI Enabled
Standard CCM Super Users
Standard CCM Server Monitoring
Standard CTI Allow Control of Phones supporting Rollover Mode
Standard CTI Allow Control of Phones supporting Connected Xfer and conf
Standard AXL API Access
Interfaces
Before initiating Testplans, Interfaces to CUCM needs to be configured. This is described in Yarnman Setup Guide under interfaces chapter https://yarnlab.atlassian.net/wiki/spaces/YSP/pages/2730393636/Yarnman+Administration#Interfaces
JTAPI Remote Setup
You will require a CTI route point with 2 DN's (lines) each with external number mask (e164). The CTI route point must be associated with the Testmate User
Click on CUCM instances then select the CUCM interface that the Remote (CTI Route Point) is on and add it
Administration
Test Plans - Overview
All test plans can be viewed on overview page (click on 'Test Plans' on the left).
Overview shows a few rows (tests), but more rows can be viewed by clicking "Load More Results" or "Load All Results" (directly underneath the overview).
Test plans can be filtered by name, customer (if linked to a certain customer), type, test, last modification user and date. To do so, click on the corresponding arrow next to the field name and select choice from the drop-down bar.
In order to see the test results in excel format, click on "Download Results" underneath the displayed tests. An excel file download will start, which you can then open directly.
Clicking on the Test plans name will bring you directly to that Test plans details.
To create a new Test plan, click on "+ Create Test Plan" in top right corner. 
Dialplan
Click on "+ Create Test Plan" in top right corner. Following page opens:
Enter a name for the new test plan.
Select requested type of test plan "Dialplan" and press 'Submit' underneath the drop-down-menu.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
After clicking 'Submit', the following page opens:
You can now specify the inputs for the test plan. Please note, that this might requires some configuration in advance (check for detailed information in the Configuration section of this document).
Select the following:
CSS Group
Number Group
Geo-Location (optional)
There are optional settings that may need to be adjusted in specific environments:
After Connect Failure
After Connect Success
JTAPI Connect Command
Pressing "Save" will save all changes made to the Dialplan test plan, that you have made.
Pressing "Refresh Tests" will update data for this very test, that might have been changed in existing CSS or Number Groups before.
Pressing "Start" will bring you to the following screen:
First, select a CUCM Instance.
After that, select the corresponding Device Pool.
Then select the corresponding Device.
Select the corresponding Line and pick a date and time for the test (once you have clicked on the 'Schedule Test-Run' field, a calendar will open and after you picked a date, take you to pick a specific time automatically).
(Pressing 'Cancel' will bring you back to the previous page, where you can select Test Timeout and Test Generation Options.)
Pressing "Start" will take you to the following screen:
You can now find your test under "Test Runs" on the very left of the page: 
A report is available for download: 
Site Migration
Click on "+ Create Test Plan" in top right corner. Following page opens: 
Enter a name for the new test plan.
Select "Site Migration" and press 'Submit' underneath the drop-down-menu.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
After clicking 'Submit', the following page opens:
First you select the corresponding Test Devices, such as CUCM Instance and Included Device Pools (one or more are possible).
You can also enter, how many devices you want to test (percentage).
You can then specify (under 'Internal Call Test Cases'), what type of call / action you want to actually test.
You can also select a Remote Endpoint (External Call Test Cases) or Media Terminator (Test Case Mixins), if required (if you do, you can choose more options for this endpoint).
You also have several other options to choose from (under Test Case Mixins), if requested.
Pressing "Save Only" will only save the Site Migration, but not do anything else.
Pressing "Cancel" will take you back to the Test Plans Overview.
"Delete Test Plan", marked in red on the right side of the page, will delete the whole Site Migration you created.
Pressing "Save and Run" will bring you to a review of your Site Migration Test (before you actually run it), that looks as follows:
You can then select a date and time for the test (once you have clicked on the 'Schedule Test-Run' field, a calendar will open and after you picked a date, take you to pick a specific time automatically).
Pressing "Cancel" will bring you back to the previous page, where you can select your Site Migration Test settings.
Pressing "Start Test-Run" will bring you to the following screen:
You can now find your test under "Test Runs" on the very left of the page: 
Results view: 
PSTN Load Test
Click on "+ Create Test Plan" in top right corner. Following page opens: 
Enter a name for the new test plan.
Select requested type of test plan and press 'Submit' underneath the drop-down-menu.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
After clicking 'Submit', the following page opens: 
Select the source and target endpoint
Number of calls
Codec
Note, that the endpoints (CTI route points) need to be added to Testmate in the CUCM instances configuration.
There are options for:
Disable DTMF identification
Continue calls on error
Timeout for call talking
Timeout for call identification
Post Verification Testing (PVT)
Click on "+ Create Test Plan" in top right corner. Following page opens: 
Enter a name for the new test plan.
Select requested type of test plan and press 'Submit' underneath the drop-down-menu.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
After clicking 'Submit', the following page opens:
You can then select your options for Internal Call Test Cases.
You can also select a Remote Endpoint, if required (if you do, you can choose more options for this endpoint).
You can also select options for Polycom VVX Phone Test Cases, Mobile Gateway Test Cases, Manual Test Cases and Test Case Mixins.
You can then select a date and time for the test (once you have clicked on the 'Schedule Test-Run' field, a calendar will open and after you picked a date, take you to pick a specific time automatically).
"Delete Test Plan", marked in red on the right side of the page, will delete the whole PTSN Load Test Plan, you created.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
Pressing "Save Only" will save the test plan, but not run it. You will already receive an error message here, if you haven't selected enough devices (minimum of 2 devices).
Select Devices to use for the test
System Acceptance Testing
Click on "+ Create Test Plan" in top right corner. Following page opens: 
Enter a name for the new test plan.
Select requested type of test plan and press 'Submit' underneath the drop-down-menu.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
After clicking 'Submit', the following page opens:
You can now select your CUCM Instance and select Included Device Pools (this option will only be selectable, once you have entered a CUCM Instance).
You can then set options for 'Internal Test Call Cases', 'External Test Call Cases' and 'Test Case Mixins'.
You can also add a Remote Endpoint. If you do so, several more options will open, which you can then choose.
"Delete Test Plan", marked in red on the right side of the page, will delete the whole SAT Test Plan, you created.
(Pressing "Cancel" will bring you back to the 'Test Plans Overview'.)
Pressing "Save Only" will save the test plan, but not run it.
Pressing "Save & Run Immediately" will run the corresponding test right away and save the corresponding test plan. It will then take you to the corresponding Test Details.
Pressing "Save & Preview" in the SAT Test Plan, will show you a preview of the SAT Test Devices.
Pressing 'Cancel' in the preview will take you back to the SAT Test Plan.
Pressing 'Regenerate Devices' in the preview will select new devices from the device pool.
Pressing "Start Test-Run" will then take you to the corresponding Test Details.
Test Run Schedules
All test run schedules can be viewed on overview page (click on 'Test Run Schedules' on the left).
Overview now shows all the schedules created.
Test Run Schedules can be filtered by name, type, total runs and next execution. To do so, click on the corresponding arrow next to the field name and select choice from the drop-down bar.
In order to see the test run schedules in excel format, click on "Download Results" underneath the displayed schedules. An excel file download will start, which you can then open directly.
Clicking on the schedules name will bring you directly to that schedule's details.
To create a new schedule, click on "+ Add Schedule" in top right corner and select the corresponding test
The following page will open:
You can now give your schedule a name and set a start and end date and time for your schedule.
You can also enter, how frequently you want your tests to run.
Under "Configuration", you can select the corresponding test plan, that you are setting the schedule for.
You can then "Save Changes" in the upper right corner (once saved, a new field "Delete Schedule" will appear on the right, where you can delete the just created schedule).
On the top right it will now also show the next execution date, that you have just set (where it said 'Not Started / Completed' before).
You can now also click on "Test Runs" on the upper left, right next to "Schedule":
Here you can see the planned Test Runs.
In order to see the planned test runs in excel format, click on "Download Results" underneath the displayed test runs. An excel file download will start, which you can then open directly.
You can also delete the planned test runs, by clicking "Delete Schedule" in the upper right corner.
Clicking on the Test runs name will bring you directly to that test runs details.
Test Run Archiving
Test Runs may be archived as required. To access go to Test Runs menu item, then select ‘Bulk Change Test Runs’
Select either specific Test Runs to archive or All Test-Runs, then select date followed by ‘Archive Test Runs’ button or ‘Delete Test Runs’ to delete selected
Archived Test Runs may be displayed by selecting ‘All’
Testmate Configuration
Configuration – CSS Groups
Click on "CSS Groups" on the menu on the left side of your screen (under "Configuration") and the following screen opens:
You can now see a list of the created CSS Groups.
In order to see the created CSS Groups in excel format, click on "Download Results" underneath the displayed CSS Groups. An excel file download will start, which you can then open directly.
CSS can be filtered by Group Name, Calling Search Spaces, Features and the date, when they were last modified. To do so, click on the corresponding arrow next to the field name and select choice from the drop-down bar.
Clicking on the CSS Groups name will bring you directly to that CSS groups details.
You can import a CSS Group by clicking "Import CSS Group" in the upper right corner. The following screen will open: 
You can now select an option and then search. Once you have searched, all the results will be displayed underneath, as per following screenshot:
Right next to 'Select an Option', you can now also choose a tenant and a department.
On default all tenants are set to "Ignore". By clicking on the word "Ignore" in front of the Name, you can select it (word 'Ignore' will change to 'Selected').
At the bottom, you can now give your new CSS Group a name. Then click on '+ Create CSS group'. The following screen will show:
Pressing "Delete Group" will delete the whole CSS Group, you just created and will bring you back to the CSS Group Overview Screen.
Pressing "Cancel" will also bring you back to the CSS Group Overview Screen, but will now show your newly created CSS Group there: 
If you now click on the name of your CSS Group, it will bring you back to the previous screen, where you have the options of "Cancel", "Delete Group" and "Submit".
Pressing "Submit" will give you the information, that the CSS Group has been successfully updated:
On the CSS Group Overview screen, you can also create a new CSS Group by clicking "+ Create CSS Group" in the upper right corner. The following screen will open: 
You can now give your CSS Group a name and then enter the Call Type Mappings yourself. Enter a name for the CSS, then enter the corresponding codes, as mentioned underneath the empty fields.
Pressing "Cancel" will bring you back to the CSS Group Overview, pressing "Delete Group" will delete the whole CSS Group and will bring you back to the CSS Group Overview Screen.
Pressing "Submit" will give you the info, that the CSS Group has been successfully created.
You can now see your newly created CSS Group in the CSS Group Overview: 
Configuration – Number Groups
Click on "Number Groups" on the menu on the left side of your screen (under "Configuration") and the following screen opens: 
You can now see a list of the created Number Groups.
In order to see the created Number Groups in excel format, click on "Download Results" underneath the displayed Number Groups. An excel file download will start, which you can then open directly.
CSS can be filtered by Label, Country and Numbers. To do so, click on the corresponding arrow next to the field name and select choice from the drop-down bar.
You can view any number group, by simply clicking on the corresponding Label.
You can now create a new Number Group by clicking "+ Create Number Group" in the upper right corner. The following screen will open: 
You can now enter a name for your new number group and the corresponding country code (i.e. GER for Germany).
You can then enter the call type, the label, the number and an corresponding geo location. By clicking on the little arrow next wo each field name, you can also select more choices for that very field (such as 'clear column', read only' etc.).
In 'Call Type' and 'Geo Location', you can select a value from a drop-down-menu. Label and number, you can enter manually (double-click in the field, to see the drop-down-menu).
Clicking on 'Cancel' will bring you back to the Number Groups Overview, without having created a number group.
Clicking on 'Submit' will bring you to the following screen: 
You can see, that your 'Number Group has been successfully created'.
Pressing "Cancel" will bring you back to the Number Group Overview, where you can now see you're created number group.
Pressing "Delete Group" will delete the whole Number Group and will bring you back to the Number Group Overview Screen.
When you are in the screen of a certain number group, you can also import and export this very number group.
Clicking on 'Import' in the upper right corner, will bring you to the following screen: 
You can find a detailed description here, of how to upload and import number groups.
Clicking on 'Export' in the upper right corner, will bring you to the following screen:
You can now click on "Download as CSV" and then you can open the corresponding file directly.
Clicking "Back" will bring you back to the corresponding number group.
Configuration – Geographic Location
Click on "Geographic Location" on the menu on the left side of your screen (under "Configuration") and the following screen opens: 
You can now see a list of the created Geographic Locations.
In order to see the created Geographic Locations in excel format, click on "Download Results" underneath the displayed Geographic Locations. An excel file download will start, which you can then open directly.
Geographic Locations can be filtered by Name, Description, Country and the date and name of the last modification. To do so, click on the corresponding arrow next to the field name and select a choice from the drop-down bar.
You can view any Geographic Location, by simply clicking on the corresponding Name.
You can now create a new Geographic Location by clicking "+ Create Geographic Location" in the upper right corner. The following screen will open: 
You can now give your Geographic Location a name and a short description and add a country.
Pressing "Cancel" will bring you back to the Geographic Location Overview.
Pressing "Submit" will tell you, that your Geographic Location has been created successfully. It will now also give you the possibility to delete it again, if you wish. See following screen: 
You can now see the created Geographic Location in the overview correspondingly.
Configuration – Codec Lists
Click on "Codec Lists" on the menu on the left side of your screen (under "Configuration") and the following screen opens:
You can now see a list of the created Codec Lists.
In order to see the created Codec Lists in excel format, click on "Download Results" underneath the displayed Geographic Locations. An excel file download will start, which you can then open directly.
Geographic Locations can be filtered by Name, Description, Codecs and the date and name of the last modification. To do so, click on the corresponding arrow next to the field name and select a choice from the drop-down bar.
You can view any Codec List, by simply clicking on the corresponding Name.
You can now create a new Codec List by clicking "+ Create Codec List" in the upper right corner. The following screen will open: 
You can now enter a name for your new Codec List and a short description.
Clicking into the 'Codecs' field will give you a drop-down-menu of different codecs to choose from.
Pressing "Cancel" will bring you back to the Codec List Overview.
Pressing "Submit" will tell you, that your Codec List has been created successfully. It will now also give you the possibility to delete it again, if you wish. See following screen:
You can now see the created codec list in the codec list overview correspondingly, see below: 
Configuration – Call Types
Click on "Call Types" on the menu on the left side of your screen (under "Configuration") and the following screen opens:
Pressing the arrows up or down, will re-order the call types and you will receive an corresponding information, as per screen below: 
To create a new Call Type, please press "+ Create Call Type" in the upper right corner. The following screen comes up:
You can now enter a name and a short description for your new call type.
Pressing 'Cancel' will bring you back to the "Call Type Overview".
Pressing 'Submit' will let you know, that you have successfully created a new Call Type.
It also gives you the option to delete the newly created Call Type (and would then bring you back to the Call Type Overview). See screen below: 
You can now see the newly created Call Type in the Call Type Overview.
Configuration – CUCM Instances
Click on "CUCM Instances" on the menu on the left side of your screen (under "Configuration") and the following screen opens (1. Left side / 2. Right side):
You can now see a list of the created CUCM Instances.
In order to see the created CUCM Instances in excel format, click on "Download Results" underneath the displayed CUCM Instances. An excel file download will start, which you can then open directly.
CUCM Instances can be filtered by Name, Description, Customer, IP-Address, User Name and Endpoints. To do so, click on the corresponding arrow next to the field name and select a choice from the drop-down bar.
You can view any CUCM Instance, by simply clicking on the corresponding Name. The following screen will open: 
You can now see the details of the CUCM Instance.
You can select a CTI Route point and add it correspondingly by hitting the "Add CTI Route Point Remote" button.
The corresponding points you just added, will automatically be saved and now be shown in the list below, as per following screen: 
Configuration – Settings
Click on "Settings" on the menu on the left side of your screen (under "Configuration") and the following screen opens: 
On this page you can now set and change the time settings for Interconnect Testing, Test Runs and Testmate. Every line has a short description underneath, what it is controlling and what is the default value for this line.
Once you have set all the values you want, you can press "Save Global Settings & Restart Effective Services".
The following screen will now show you, that you have set changes and that all services will be restarted correspondingly: