Account Prerequisites for Cloud Video

test_mate Cloud Video requires a Webex integration account, a Webex end-user account and a Microsoft Graph App account.

The environment should be already configured with the features from Webex and Microsoft to be tested. e.g. CVI, Calendar resources, xApi

It's recommended the Webex Enduser account be the same as the calendar user for scheduling.

Webex end-user account

Create or use an existing Webex user account

add the following Licenses and Permissions

Webex Licenses

Webex Administrator roles

Basic Messaging

Organizational: Read-only administrator

Basic Space Meetings

Services: Webex Read-only Administrator

Webex Meetings

Webex Integration Account

Please note the Client secret, as it won't be shown again.

Permissions in test_mate Webex API v2.5.17

Webex Permission enabled

Scope

(blue star)

meeting:schedules_read

(blue star)

meeting:schedules_write

(blue star)

meeting:recordings_read

(blue star)

meeting:recordings_write

meeting:preferences_read

meeting:preferences_write

(blue star)

meeting:controls_read

(blue star)

meeting:controls_write

(blue star)

meeting:participants_read

(blue star)

meeting:participants_write

(blue star)

meeting:admin_participants_read

spark-admin:telephony_config_read

spark-admin:telephony_config_write

(blue star)

meeting:admin_schedule_read

(blue star)

meeting:admin_schedule_write

meeting:admin_recordings_read

meeting:admin_recordings_write

(blue star)

meeting:admin_transcripts_read

meeting:admin_preferences_write

meeting:admin_preferences_read

spark-compliance:meetings_read

(blue star)

meeting:transcripts_read

spark-compliance:meetings_write

(blue star)

spark-admin:workspace_locations_read

spark-admin:workspace_locations_write

spark-admin:workspace_metrics_read

spark-admin:broadworks_billing_reports_read

spark-admin:broadworks_billing_reports_write

spark:allFull

(blue star)

spark:calls_read

(blue star)

spark:devices_read

spark:devices_write

spark:memberships_read

spark:memberships_write

spark:messages_read

spark:messages_write

(blue star)

spark:organizations_read

(blue star)

spark:people_read

spark:people_write

(blue star)

spark:places_read

spark:places_write

(blue star)

spark:rooms_read

spark:rooms_write

spark:team_memberships_read

spark:team_memberships_write

spark:teams_read

spark:teams_write

(blue star)

spark:xapi_statuses

(blue star)

spark:xapi_commands

(blue star)

spark:xsi

spark-admin:devices_read

spark-admin:devices_write

(blue star)

spark-admin:licenses_read

(blue star)

spark-admin:organizations_read

spark-admin:organizations_write

(blue star)

spark-admin:people_read

spark-admin:people_write

(blue star)

spark-admin:places_read

spark-admin:places_write

(blue star)

spark-admin:resource_group_memberships_read

spark-admin:resource_group_memberships_write

(blue star)

spark-admin:resource_groups_read

(blue star)

spark-admin:roles_read

(blue star)

spark-admin:call_qualities_read

(blue star)

spark-admin:workspaces_read

spark-admin:workspaces_write

(blue star)

spark-admin:xsi

(blue star)

spark-compliance:events_read

(blue star)

spark-compliance:memberships_read

spark-compliance:memberships_write

(blue star)

spark-compliance:messages_read

spark-compliance:messages_write

(blue star)

spark-compliance:rooms_read

spark-compliance:rooms_write

(blue star)

spark-compliance:team_memberships_read

spark-compliance:team_memberships_write

(blue star)

spark-compliance:teams_read

(blue star)

spark-admin:broadworks_enterprises_read

(blue star)

identity:placeonetimepassword_create

(blue star)

Identity:one_time_password

spark-compliance:webhooks_write

(blue star)

spark-compliance:webhooks_read

spark:calls_write

(blue star)

spark-admin:wholesale_billing_reports_read

(blue star)

spark-admin:hybrid_clusters_read

spark-admin:wholesale_billing_reports_write

(blue star)

spark-admin:hybrid_connectors_read

(blue star)

spark-admin:broadworks_subscribers_read

spark-admin:broadworks_subscribers_write

(blue star)

analytics:read_all

(blue star)

audit:events_read

spark-admin:wholesale_customers_write

spark-admin:wholesale_customers_read

spark-admin:wholesale_subscribers_write

spark-admin:wholesale_subscribers_read

identity:tokens_read

identity:tokens_write

spark-admin:broadworks_enterprises_write

Webex Edge for Devices

Required for non-cloud registered devices.

This allows Webex Control Hub to control on-prem/HCS controlled devices for xApi commands. A desktop application must be downloaded and configured to link the on-prem unified communication services with the Webex control hub. A new workspace for each device will be created with the device name.

refer to the following link for more information on setup and configuration.

https://help.webex.com/en-us/article/cy2l2z/Webex-Edge-for-Devices

Microsoft Application (API) Account

Create a new application registration in Azure AD with the following API permissions (TBC)

Redirect URI isn't required

Permissions in test_mate Microsoft Graph API v2.5.17

https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps

Microsoft Graph

Type

Calendars.Read

Application

Calendars.ReadWrite

Application

OnlineMeetingArtifact.Read.All

Application

OnlineMeetings.Read.All

Application

OnlineMeetings.ReadWrite.All

Application

User.Read.All

Application

CallRecords.Read.All

Application

Create a Client Secret ID with the duration timeframe per your security standards.

Please note the Client Secret ID Value, as it won't be shown again.

Webex interface setup

Log in to Yarnman, open Administration App-> Interfaces.

From the top right corner, click +Add interface, select Interface type Webex API (OAuth2 - Cloud)

Define Interface Name, Description, then click Submit.

Internet Proxy Configuration

A proxy may be required if test_mate doesn’t have direct access to the internet for API calls. test_mate supports the following authentication modes.

To configure the proxy, toggle the Enable Proxy to ON, and provide the HTTP Proxy details, which will be used for both HTTP/S.

Arm Bindings and Webex Credentials

note

Firefox is required to perform the OAuth part of the Webex credentials.

Firefox is required to perform the OAuth part of the Webex credentials.

Select Node to bind the interface and click Bind to Arm

Note the Redirect URL; This will be needed in the credentials settings and the Webex integration portal.

Click Update / Set Credentials to enter the Webex details.

Enter in the Application name, Client ID and the Redirect URL from the Arm Binding page and click Submit

you will be presented with the Webex OAuth login flow; provide the Webex user details that will be used for test_mate

The test_mate Webex user will be used to schedule and admit participants into meetings and should be the same user as the Microsoft graph interface.

If prompted, accept the Webex permissions to be granted.

You will be redirected back to test_mate to re-authenticate.

Finally, add the Client Secret provided with the Webex app setup and click submit.

Test Webex connection

Click the Test Connection button. This will return the Webex user details that granted the Webex integration.

Microsoft Graph interface setup

Log in to Yarnman, open Administration App-> Interfaces.

From the top right corner, click +Add interface, select Interface type Microsoft Graph API (OAuth2 - Cloud) / Azure AD.

Define Interface Name, Description, and Description, then click Submit.

Internet Proxy Configuration

A proxy may be required if test_mate doesn’t have direct access to the internet for the interface API calls. test_mate supports the following authentication modes.

To configure the proxy, toggle the Enable Proxy to ON. Provide the HTTP Proxy details which will be used for both HTTP/S.

Arm Bindings and Microsoft Graph Credentials

Select Node to bind the interface and click Bind to Arm

Click Update / Set Credentials to enter the Microsoft graph details.

Enter in the App ID, App Key and Default TenantID click Submit

Test Microsoft Graph connection

Click the Test Connection button. This will return the top 5 Users in the tenant ID.

Static Dial-In Test Plan

This type of test allows two or more endpoints to call into a static/predefined meeting with or without DTMF meeting details (Host/Guest ID and Pins). This would typically be used with an external 3rd party bridge.

Create Test Plan

Log in to Yarnman, open test_mate App->Test Plans.

From the top right corner, click +Create Test Plan, select type Cloud-Video static Dial-In.

Define a Test Plan Name and click Submit.

Endpoints

Select the Webex account/interface that was previously configured, and click Sync Webex Now if required.

Click Find Endpoints to select devices used within the test plan and click Done.

Depending on the destination being called, select the Dial Method, Meeting ID and Pin if required,

The dial will use the SIP and Webex. Join will try and use the Spark/Webex protocol to join.

Default is Dial

Define meeting settings as required, followed by either Save Only to run the test later or Save & Run to execute the test now.

Static Meeting Settings

Setting

Description

Name

Name of the test plan.

Destination

sip URI or PhoneNumber.

Force Existing Call Teardown

test_mate will end any active calls on the endpoints before executing the test plan.

Mute Devices During Test

test_mate will signal to Mute the Microphone on the endpoint during the test.

Pause Before MeetingID (secs)

The number of seconds test_mate waits before sending DTMF digits for the MeetingID after the call Connects.

Pause Before PIN (secs)

The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects.

Webex Account / Interface

Select the Webex account/interface used to control the endpoints.

Endpoints Find Endpoints

Launches endpoint selector window.

Endpoints Sync Webex Now

Synchronises endpoints for any new or removed endpoints.

Endpoint Quality Monitoring

Enable or Disable Quality monitoring.

Packet Statistics Wait (secs)

the number of seconds the call is left connected to allow statistics to be collected

Test Content Sharing

TBC

Schedule Test Run

TBC

Scheduled Calendar invite Test Plan

This test plan simulates creating a scheduled meeting via a calendar invite and triggering the scheduling associated with the calendar user.

High-level test plan.

Create Meeting -> Check endpoints accepted -> Dial endpoints into meeting -> Check stats -> End calls -> After meeting cleanup and collection of cloud stats if available.

Scheduled Common Meeting Settings

Setting

Description

Name

Name of the test plan.

Meeting Name

The subject of the meeting (displayed in the calendar invite)

Meeting Method

Select the meeting integration to be used, e.g. Microsoft Teams with CVI.

Meeting Location

Depending on the integration, the location field within the meeting invite may be required to have a specific format/value.

Force Existing Call Teardown

test_mate will end any active calls on the endpoints before executing the test plan.

Mute Devices During Test

test_mate will signal to Mute the Microphone on the endpoint during the test.

Removing Meeting After Test

After the test plan has been completed, remove the calendar invite. This is useful to remove any room reservations and clean up the calendar.

Meeting Create Then Start Pause (secs)

The number of seconds test_mate waits before creating the meeting invite when the test is executed.

Max Waiting Time For Device Acceptance (secs)

The number of seconds test_mate for meeting rooms to accept the invite.

Calendar Check Polling Interval For Acceptance (secs)

The number of seconds test_mate checks in each interval to accept the calendar invite.

Max Waiting Time For Meeting Body Update (secs)

The number of seconds test_mate waits for the meeting to be updated with the SIP URI details required to make the video call.

Polling Interval For Meeting Body Update (secs)

The number of seconds test_mate checks for the SIP URI details in each interval.

Pause Before PIN (secs)

The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects.

Wait Before Loading Webex Meeting (secs)

The number of seconds test_mate waits before validating each endpoint has joined the meeting and being admitted (if supported by the meeting type) successfully.

Webex XAPI Request Pause After Error (secs)

Test_mate waits for the number of seconds test_mate waits after receiving a Xapi error to an endpoint command.

Webex XAPI Request Retries After Error

The maximum number of retries test_mate attempts to a Xapi command.

Microsoft Graph Interface

Select the Microsoft graph application registration user.

Calendar User

Selects the Calendar user to schedule meetings for testing.

Webex Account / Interface

Select the Webex account/interface used to control the endpoints.

Endpoints Find Endpoints

Launches endpoint selector window.

Endpoints Sync Webex Now

Synchronises endpoints for any new or removed endpoints.

Endpoint Quality Monitoring

Enable or Disable Quality monitoring.

Packet Statistics Wait (secs)

the number of seconds the call is left connected to allow statistics to be collected

Test Content Sharing

TBC

Schedule Test Run

TBC

Cisco CVI / Microsoft Teams meeting

Setting

Description

Collect Attendance Reports

Collects meeting qualities if available from the provider.

Wait After Meeting For Attendance Reports (secs)

Number of seconds test_mate waits before retrieving stats from the provider.

Cisco Webex using @meet @webex

Setting

Description

Location @meet

@meet is a keyword that triggers a Webex integration within the calendaring service, which uses the Webex personal meeting room for the invite.

Location @webex

@webex creates a one-time meeting with associated Webex space for the invite.

Collect Webex Stats Report

Collects meeting qualities if available from the provider.

Wait After Meeting For Webex Stats (secs)

Number of seconds test_mate waits before retrieving stats from the provider.

refer to the following link for the Microsoft Webex location integration https://help.webex.com/en-us/article/nwtulzs/Webex-App-%7C-Schedule-a-meeting-using-@webex-or-@meet

Cloud Video Test Runs Results

Webex Edge for Devices - OnPrem/HCS registered

On-prem registered devices can be controlled via Webex Edge for Devices, allowing xApi commands to be proxied securely to the endpoint via Webex. It's also possible to provide enhanced/optimised Webex meetings to On-prem devices whilst retaining their existing call flows.

Refer to the following for further information.

https://help.webex.com/en-us/article/cy2l2z/Webex-Edge-for-Devices

https://www.cisco.com/c/dam/en/us/td/docs/voice_ip_comm/cloudCollaboration/wbxt/whitepapers/webex-edge-devices-white-paper.pdf

Webex Call routing

Cloud Registered

Video calls will route within Webex as normal

Webex Call routing - Sip

Video calls will route via the existing signalling path, e.g. via an expressway to the internet to Webex.

This is the default setting with Webex Edge for Devices.

Webex Call routing - Webex optimised

Video calls to Webex will route directly from the Endpoint to the internet, e.g. not via expressways. This requires extra configuration to enable this feature and possible network/firewall changes. Non-Webex Video calls will route via the existing signalling path, e.g. via an expressway.

Troubleshooting

API errors

Problem: Blank page after submitting Webex credentials

Suggested course of action:

Webex requires the request permission from test_mate to match the configured permissions in Webex. Else the integration will fail.

Problem: Expired credentials/tokens in Webex

Suggested course of action:

Problem: Missing Redirect URL for Webex integration account

Suggested course of action :

Deleting and re-adding an interface will create a new unique redirect URL

Problem: Microsoft Graph ErrorAccessDenied

Suggested course of action :

Your organisation may limit access to all mailboxes/calendars. Check with your Azure/o365 admin

https://docs.microsoft.com/en-us/graph/auth-limit-mailbox-access#configure-applicationaccesspolicy

Problem: Webex Device access error

Suggested course of action :

If your organisation won't allow admin read-only access to Webex, The integration user needs to be added to every workspace used for test_mate device testing.

Internet / Proxy Errors

Problem: Wrong Protocol

Suggested course of action:

Problem: Wrong User/Password

Suggested course of action:

Problem: DNS access and/or internet access are unavailable