Test_mate Cloud Video

Owned by Glen Steedman
Last updated: Nov 30, 2023
10 min read

Account Prerequisites for Cloud Video

test_mate Cloud Video requires a Webex integration account, a Webex end-user account and a Microsoft Graph App account.

The environment should be already configured with the features from Webex and Microsoft to be tested. e.g. CVI, Calendar resources, xApi

It's recommended the Webex Enduser account be the same as the calendar user for scheduling.

Webex end-user account

Create or use an existing Webex user account

add the following Licenses and Permissions

Webex Licenses

Webex Administrator roles

Webex Licenses

Webex Administrator roles

Basic Messaging

Organizational: Read-only administrator

Basic Space Meetings

Services: Webex Read-only Administrator

Webex Meetings

 

Webex Integration Account

  • Create a new Webex integration app at the Webex developer portal https://developer.webex.com/my-apps

  • Select No to Mobile SDK

  • Provide an integration name, logo

  • add the redirect URL from the test_mate Webex interface

  • Set the permissions to the following

Please note the Client secret, as it won't be shown again.

Permissions in test_mate Webex API v2.5.24 - Note that new scopes may be added by Cisco

Yarnman Powered by Photon ( V 2.6.2 and higher )has a different set of scopes and configuration , Refer to Yarnman (YM-PH) Interfaces Below is a reference only for previous versions for webex permissions.

Webex Permission enabled

Scope

Webex Permission enabled

Scope

meeting:schedules_read

meeting:schedules_write

meeting:recordings_read

meeting:recordings_write

meeting:controls_read

meeting:controls_write

meeting:participants_read

meeting:participants_write

meeting:admin_participants_read

meeting:admin_schedule_read

meeting:admin_schedule_write

meeting:admin_transcripts_read

meeting:transcripts_read

spark-admin:workspace_locations_read

spark:calls_read

spark:devices_read

spark:organizations_read

spark:people_read

spark:places_read

spark:rooms_read

spark:xapi_statuses

spark:xapi_commands

spark:xsi

spark-admin:licenses_read

spark-admin:organizations_read

spark-admin:people_read

spark-admin:places_read

spark-admin:resource_group_memberships_read

spark-admin:resource_groups_read

spark-admin:roles_read

spark-admin:call_qualities_read

spark-admin:workspaces_read

spark-compliance:events_read

spark-compliance:memberships_read

spark-compliance:messages_read

spark-compliance:rooms_read

spark-compliance:team_memberships_read

spark-compliance:teams_read

spark-admin:broadworks_enterprises_read

identity:placeonetimepassword_create

Identity:one_time_password

spark-compliance:webhooks_read

spark-admin:wholesale_billing_reports_read

spark-admin:hybrid_clusters_read

spark-admin:hybrid_connectors_read

spark-admin:broadworks_subscribers_read

analytics:read_all

audit:events_read

 

Webex Edge for Devices

Required for non-cloud registered devices.

This allows Webex Control Hub to control on-prem/HCS controlled devices for xApi commands. A desktop application must be downloaded and configured to link the on-prem unified communication services with the Webex control hub. A new workspace for each device will be created with the device name.

refer to the following link for more information on setup and configuration.

https://help.webex.com/en-us/article/cy2l2z/Webex-Edge-for-Devices

Microsoft Application (API) Account

Create a new application registration in Azure AD with the following API permissions (TBC)

Permissions in test_mate Microsoft Graph API v2.5.17

https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps

Microsoft Graph

Type

Microsoft Graph

Type

Calendars.Read

Application

Calendars.ReadWrite

Application

OnlineMeetingArtifact.Read.All

Application

OnlineMeetings.Read.All

Application

OnlineMeetings.ReadWrite.All

Application

User.Read.All

Application

CallRecords.Read.All

Application

Create a Client Secret ID with the duration timeframe per your security standards.

Webex interface setup

Log in to Yarnman, open Administration App-> Interfaces.

From the top right corner, click +Add interface, select Interface type Webex API (OAuth2 - Cloud)

Define Interface Name, Description, then click Submit.

Internet Proxy Configuration

A proxy may be required if test_mate doesn’t have direct access to the internet for API calls. test_mate supports the following authentication modes.

  • No Authentication

  • Basic Authentication

  • Digest Authentication

To configure the proxy, toggle the Enable Proxy to ON, and provide the HTTP Proxy details, which will be used for both HTTP/S.

Arm Bindings and Webex Credentials

Select Node to bind the interface and click Bind to Arm

Click Update / Set Credentials to enter the Webex details.

Enter in the Application name, Client ID and the Redirect URL from the Arm Binding page and click Submit

you will be presented with the Webex OAuth login flow; provide the Webex user details that will be used for test_mate

If prompted, accept the Webex permissions to be granted.

You will be redirected back to test_mate to re-authenticate.

Finally, add the Client Secret provided with the Webex app setup and click submit.

Test Webex connection

Click the Test Connection button. This will return the Webex user details that granted the Webex integration.

 

Microsoft Graph interface setup

Log in to Yarnman, open Administration App-> Interfaces.

From the top right corner, click +Add interface, select Interface type Microsoft Graph API (OAuth2 - Cloud) / Azure AD.

Define Interface Name, Description, and Description, then click Submit.

Internet Proxy Configuration

A proxy may be required if test_mate doesn’t have direct access to the internet for the interface API calls. test_mate supports the following authentication modes.

  • No Authentication

  • Basic Authentication

  • Digest Authentication

To configure the proxy, toggle the Enable Proxy to ON. Provide the HTTP Proxy details which will be used for both HTTP/S.

Arm Bindings and Microsoft Graph Credentials

Select Node to bind the interface and click Bind to Arm

Click Update / Set Credentials to enter the Microsoft graph details.

Enter in the App ID, App Key and Default TenantID click Submit

Test Microsoft Graph connection

Click the Test Connection button. This will return the top 5 Users in the tenant ID.

 

Static Dial-In Test Plan

This type of test allows two or more endpoints to call into a static/predefined meeting with or without DTMF meeting details (Host/Guest ID and Pins). This would typically be used with an external 3rd party bridge.

 

Create Test Plan

Log in to Yarnman, open test_mate App->Test Plans.

From the top right corner, click +Create Test Plan, select type Cloud-Video static Dial-In.

Define a Test Plan Name and click Submit.

Endpoints

Select the Webex account/interface that was previously configured, and click Sync Webex Now if required.

Click Find Endpoints to select devices used within the test plan and click Done.

Depending on the destination being called, select the Dial Method, Meeting ID and Pin if required,

 

Define meeting settings as required, followed by either Save Only to run the test later or Save & Run to execute the test now.

Static Meeting Settings

Setting

Description

Setting

Description

Name

Name of the test plan.

Destination

sip URI or PhoneNumber.

Force Existing Call Teardown

test_mate will end any active calls on the endpoints before executing the test plan.

Mute Devices During Test

test_mate will signal to Mute the Microphone on the endpoint during the test.

Pause Before MeetingID (secs)

The number of seconds test_mate waits before sending DTMF digits for the MeetingID after the call Connects.

Pause Before PIN (secs)

The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects.

Webex Account / Interface

Select the Webex account/interface used to control the endpoints.

Endpoints Find Endpoints

Launches endpoint selector window.

Endpoints Sync Webex Now

Synchronises endpoints for any new or removed endpoints.

Endpoint Quality Monitoring

Enable or Disable Quality monitoring.

Packet Statistics Wait (secs)

the number of seconds the call is left connected to allow statistics to be collected

Test Content Sharing

TBC

Schedule Test Run

TBC

Scheduled Calendar invite Test Plan

This test plan simulates creating a scheduled meeting via a calendar invite and triggering the scheduling associated with the calendar user.

High-level test plan.

Create Meeting -> Check endpoints accepted -> Dial endpoints into meeting -> Check stats -> End calls -> After meeting cleanup and collection of cloud stats if available.

Scheduled Common Meeting Settings

Setting

Description

Setting

Description

Name

Name of the test plan.

Meeting Name

The subject of the meeting (displayed in the calendar invite)

Meeting Method

Select the meeting integration to be used, e.g. Microsoft Teams with CVI.

Meeting Location

Depending on the integration, the location field within the meeting invite may be required to have a specific format/value.

Force Existing Call Teardown

test_mate will end any active calls on the endpoints before executing the test plan.

Mute Devices During Test

test_mate will signal to Mute the Microphone on the endpoint during the test.

Removing Meeting After Test

After the test plan has been completed, remove the calendar invite. This is useful to remove any room reservations and clean up the calendar.

Meeting Create Then Start Pause (secs)

The number of seconds test_mate waits before creating the meeting invite when the test is executed.

Max Waiting Time For Device Acceptance (secs)

The number of seconds test_mate for meeting rooms to accept the invite.

Calendar Check Polling Interval For Acceptance (secs)

The number of seconds test_mate checks in each interval to accept the calendar invite.

Max Waiting Time For Meeting Body Update (secs)

The number of seconds test_mate waits for the meeting to be updated with the SIP URI details required to make the video call.

Polling Interval For Meeting Body Update (secs)

The number of seconds test_mate checks for the SIP URI details in each interval.

Pause Before PIN (secs)

The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects.

Wait Before Loading Webex Meeting (secs)

The number of seconds test_mate waits before validating each endpoint has joined the meeting and being admitted (if supported by the meeting type) successfully.

Webex XAPI Request Pause After Error (secs)

Test_mate waits for the number of seconds test_mate waits after receiving a Xapi error to an endpoint command.

Webex XAPI Request Retries After Error

The maximum number of retries test_mate attempts to a Xapi command.

Microsoft Graph Interface

Select the Microsoft graph application registration user.

Calendar User

Selects the Calendar user to schedule meetings for testing.

Webex Account / Interface

Select the Webex account/interface used to control the endpoints.

Endpoints Find Endpoints

Launches endpoint selector window.

Endpoints Sync Webex Now

Synchronises endpoints for any new or removed endpoints.

Endpoint Quality Monitoring

Enable or Disable Quality monitoring.

Packet Statistics Wait (secs)

the number of seconds the call is left connected to allow statistics to be collected

Test Content Sharing

TBC

Schedule Test Run

TBC

Cisco CVI / Microsoft Teams meeting

  • Endpoints are available to make a call.

  • Apply specific meeting settings (e.g. mute endpoints before calling).

  • Creates an online meeting via the Microsoft Graph interface (simulates an end-user creating the meeting in outlook).

  • Endpoints have accepted or declined the meeting in their calendar.

  • Dials the CVI bridge number that is dynamically generated with the online meeting.

  • Joins all endpoints to the call and confirms they have connected.

  • Collects meeting participants and meeting qualities if available from the provider.

  • Disconnect the endpoints

  • Any other after meetings tasks (e.g. Delete meeting invite).

 

Setting

Description

Setting

Description

Collect Attendance Reports

Collects meeting qualities if available from the provider.

Wait After Meeting For Attendance Reports (secs)

Number of seconds test_mate waits before retrieving stats from the provider.

 

 

Cisco Webex using @meet @webex

  • Endpoints are available to make a call.

  • Apply specific meeting settings (e.g. mute endpoints before calling).

  • Creates a regular meeting (e.g. without selecting teams) via the Microsoft Graph interface (simulates an end-user creating the meeting in outlook).

  • Endpoints have accepted or declined the meeting in their calendar.

  • Dials the Webex @webex or @meet details dynamically generated by Webex parsing the location field.

  • Joins all endpoints to the call and confirms they have connected.

  • Collects meeting participants and meeting qualities if available from the provider.

  • Disconnect the endpoints

  • Any other after meetings tasks (e.g. Delete meeting invite).

 

Setting

Description

Setting

Description

Location @meet

@meet is a keyword that triggers a Webex integration within the calendaring service, which uses the Webex personal meeting room for the invite.

Location @webex

@webex creates a one-time meeting with associated Webex space for the invite.

Collect Webex Stats Report

Collects meeting qualities if available from the provider.

Wait After Meeting For Webex Stats (secs)

Number of seconds test_mate waits before retrieving stats from the provider.

refer to the following link for the Microsoft Webex location integration https://help.webex.com/en-us/article/nwtulzs/Webex-App-%7C-Schedule-a-meeting-using-@webex-or-@meet

Cloud Video Test Runs Results

Webex Edge for Devices - OnPrem/HCS registered

On-prem registered devices can be controlled via Webex Edge for Devices, allowing xApi commands to be proxied securely to the endpoint via Webex. It's also possible to provide enhanced/optimised Webex meetings to On-prem devices whilst retaining their existing call flows.

Refer to the following for further information.

Webex Call routing

Cloud Registered

Video calls will route within Webex as normal

Webex Call routing - SIP

Video calls will route via the existing signalling path, e.g. via an expressway to the internet to Webex.

This is the default setting with Webex Edge for Devices.

Webex Call routing - Webex optimised

Video calls to Webex will route directly from the Endpoint to the internet, e.g. not via expressways. This requires extra configuration to enable this feature and possible network/firewall changes. Non-Webex Video calls will route via the existing signalling path, e.g. via an expressway.

Troubleshooting

API errors

Problem: Blank page after submitting Webex credentials

 

Suggested course of action:

  • Check permissions for the integration account match in test_mate and Webex.

  • Try using an alternate browser such as Firefox.

 

Problem: Expired credentials/tokens in Webex

Suggested course of action:

  • Click on Update / Set Credentials to reauthorise with the account that grated the Webex integration and/or check the correct details supplied.

 

Problem: Missing Redirect URL for Webex integration account

 

Suggested course of action :

  • Ensure the Redirect URL is the same as displayed within the test_mate interface.

  • Remove any spaces from the URL's beginning and/or end when using copy/paste to Webex.

Problem: Microsoft Graph ErrorAccessDenied

 

Suggested course of action :

  • Check the correct application account/client id used with the proper tenant id.

  • Check application account/client id has access to mailboxes/calendars selected for test plan(s).

  • Check API permissions in azure AD setup correctly and granted.

 

Problem: Webex Device access error

 

 

Suggested course of action :

  • Check Webex admin permissions for the Webex end-user scheduling the meeting.

  • Check device is online and registered.

 

Internet / Proxy Errors

Problem: Wrong Protocol

Suggested course of action:

  • Check the correct protocol has been specified, e.g. HTTP

  • Check the correct port number provided

Problem: Wrong User/Password

Suggested course of action:

  • Check correct username / password entered ( password could be case sensitive depending on the 3rd party proxy).

  • Check Proxy allows access to the domains required for API calls to the providers.

Problem: DNS access and/or internet access are unavailable

 

yarnlab website