Account Prerequisites for Cloud Video
test_mate Cloud Video requires a Webex integration account, a Webex end-user account and a Microsoft Graph App account.
The environment should be already configured with the features from Webex and Microsoft to be tested. e.g. CVI, Calendar resources, xApi
It's recommended the Webex Enduser account be the same as the calendar user for scheduling.
Webex end-user account
Create or use an existing Webex user account
add the following Licenses and Permissions
Webex Licenses | Webex Administrator roles |
---|---|
Basic Messaging | Organizational: Read-only administrator |
Basic Space Meetings | Services: Webex Read-only Administrator |
Webex Meetings |
Webex Integration Account
Create a new Webex integration app at the Webex developer portal https://developer.webex.com/my-apps
Select No to Mobile SDK
Provide an integration name, logo
add the redirect URL from the test_mate Webex interface
Set the permissions to the following
Please note the Client secret, as it won't be shown again.
Permissions in test_mate Webex API v2.5.17
Webex Permission enabled | Scope |
---|---|
☑ | meeting:schedules_read |
☑ | meeting:schedules_write |
☑ | meeting:recordings_read |
☑ | meeting:recordings_write |
meeting:preferences_read | |
meeting:preferences_write | |
☑ | meeting:controls_read |
☑ | meeting:controls_write |
☑ | meeting:participants_read |
☑ | meeting:participants_write |
☑ | meeting:admin_participants_read |
spark-admin:telephony_config_read | |
spark-admin:telephony_config_write | |
☑ | meeting:admin_schedule_read |
☑ | meeting:admin_schedule_write |
meeting:admin_recordings_read | |
meeting:admin_recordings_write | |
☑ | meeting:admin_transcripts_read |
meeting:admin_preferences_write | |
meeting:admin_preferences_read | |
spark-compliance:meetings_read | |
☑ | meeting:transcripts_read |
spark-compliance:meetings_write | |
☑ | spark-admin:workspace_locations_read |
spark-admin:workspace_locations_write | |
spark-admin:workspace_metrics_read | |
spark-admin:broadworks_billing_reports_read | |
spark-admin:broadworks_billing_reports_write | |
spark:allFull | |
☑ | spark:calls_read |
☑ | spark:devices_read |
spark:devices_write | |
spark:memberships_read | |
spark:memberships_write | |
spark:messages_read | |
spark:messages_write | |
☑ | spark:organizations_read |
☑ | spark:people_read |
spark:people_write | |
☑ | spark:places_read |
spark:places_write | |
☑ | spark:rooms_read |
spark:rooms_write | |
spark:team_memberships_read | |
spark:team_memberships_write | |
spark:teams_read | |
spark:teams_write | |
☑ | spark:xapi_statuses |
☑ | spark:xapi_commands |
☑ |
|
spark-admin:devices_read | |
spark-admin:devices_write | |
☑ | spark-admin:licenses_read |
☑ | spark-admin:organizations_read |
spark-admin:organizations_write | |
☑ | spark-admin:people_read |
spark-admin:people_write | |
☑ | spark-admin:places_read |
spark-admin:places_write | |
☑ | spark-admin:resource_group_memberships_read |
spark-admin:resource_group_memberships_write | |
☑ | spark-admin:resource_groups_read |
☑ | spark-admin:roles_read |
☑ | spark-admin:call_qualities_read |
☑ | spark-admin:workspaces_read |
spark-admin:workspaces_write | |
☑ |
|
☑ | spark-compliance:events_read |
☑ | spark-compliance:memberships_read |
spark-compliance:memberships_write | |
☑ | spark-compliance:messages_read |
spark-compliance:messages_write | |
☑ | spark-compliance:rooms_read |
spark-compliance:rooms_write | |
☑ | spark-compliance:team_memberships_read |
spark-compliance:team_memberships_write | |
☑ | spark-compliance:teams_read |
☑ | spark-admin:broadworks_enterprises_read |
☑ | identity:placeonetimepassword_create |
☑ | Identity:one_time_password |
spark-compliance:webhooks_write | |
☑ | spark-compliance:webhooks_read |
spark:calls_write | |
☑ | spark-admin:wholesale_billing_reports_read |
☑ | spark-admin:hybrid_clusters_read |
spark-admin:wholesale_billing_reports_write | |
☑ | spark-admin:hybrid_connectors_read |
☑ | spark-admin:broadworks_subscribers_read |
spark-admin:broadworks_subscribers_write | |
☑ | analytics:read_all |
☑ | audit:events_read |
spark-admin:wholesale_customers_write | |
spark-admin:wholesale_customers_read | |
spark-admin:wholesale_subscribers_write | |
spark-admin:wholesale_subscribers_read | |
identity:tokens_read | |
identity:tokens_write | |
spark-admin:broadworks_enterprises_write |
Webex Edge for Devices
Required for non-cloud registered devices.
This allows Webex Control Hub to control on-prem/HCS controlled devices for xApi commands. A desktop application must be downloaded and configured to link the on-prem unified communication services with the Webex control hub. A new workspace for each device will be created with the device name.
refer to the following link for more information on setup and configuration.
https://help.webex.com/en-us/article/cy2l2z/Webex-Edge-for-Devices
Microsoft Application (API) Account
Create a new application registration in Azure AD with the following API permissions (TBC)
Redirect URI isn't required
Permissions in test_mate Microsoft Graph API v2.5.17
https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps
Microsoft Graph | Type |
---|---|
Calendars.Read | Application |
Calendars.ReadWrite | Application |
OnlineMeetingArtifact.Read.All | Application |
OnlineMeetings.Read.All | Application |
OnlineMeetings.ReadWrite.All | Application |
User.Read.All | Application |
CallRecords.Read.All | Application |
Create a Client Secret ID with the duration timeframe per your security standards.
Please note the Client Secret ID Value, as it won't be shown again.
Webex interface setup
Log in to Yarnman, open Administration App-> Interfaces.
From the top right corner, click +Add interface, select Interface type Webex API (OAuth2 - Cloud)
Define Interface Name, Description, then click Submit.
Internet Proxy Configuration
A proxy may be required if test_mate doesn’t have direct access to the internet for API calls. test_mate supports the following authentication modes.
No Authentication
Basic Authentication
Digest Authentication
To configure the proxy, toggle the Enable Proxy to ON, and provide the HTTP Proxy details, which will be used for both HTTP/S.
Arm Bindings and Webex Credentials
Firefox is required to perform the OAuth part of the Webex credentials.
Select Node to bind the interface and click Bind to Arm
Note the Redirect URL; This will be needed in the credentials settings and the Webex integration portal.
Click Update / Set Credentials to enter the Webex details.
Enter in the Application name, Client ID and the Redirect URL from the Arm Binding page and click Submit
you will be presented with the Webex OAuth login flow; provide the Webex user details that will be used for test_mate
The test_mate Webex user will be used to schedule and admit participants into meetings and should be the same user as the Microsoft graph interface.
If prompted, accept the Webex permissions to be granted.
You will be redirected back to test_mate to re-authenticate.
Finally, add the Client Secret provided with the Webex app setup and click submit.
Test Webex connection
Click the Test Connection button. This will return the Webex user details that granted the Webex integration.
Microsoft Graph interface setup
Log in to Yarnman, open Administration App-> Interfaces.
From the top right corner, click +Add interface, select Interface type Microsoft Graph API (OAuth2 - Cloud) / Azure AD.
Define Interface Name, Description, and Description, then click Submit.
Internet Proxy Configuration
A proxy may be required if test_mate doesn’t have direct access to the internet for the interface API calls. test_mate supports the following authentication modes.
No Authentication
Basic Authentication
Digest Authentication
To configure the proxy, toggle the Enable Proxy to ON. Provide the HTTP Proxy details which will be used for both HTTP/S.
Arm Bindings and Microsoft Graph Credentials
Select Node to bind the interface and click Bind to Arm
Click Update / Set Credentials to enter the Microsoft graph details.
Enter in the App ID, App Key and Default TenantID click Submit
Test Microsoft Graph connection
Click the Test Connection button. This will return the top 5 Users in the tenant ID.
Static Dial-In Test Plan
This type of test allows two or more endpoints to call into a static/predefined meeting with or without DTMF meeting details (Host/Guest ID and Pins). This would typically be used with an external 3rd party bridge.
Create Test Plan
Log in to Yarnman, open test_mate App->Test Plans.
From the top right corner, click +Create Test Plan, select type Cloud-Video static Dial-In.
Define a Test Plan Name and click Submit.
Endpoints
Select the Webex account/interface that was previously configured, and click Sync Webex Now if required.
Click Find Endpoints to select devices used within the test plan and click Done.
Depending on the destination being called, select the Dial Method, Meeting ID and Pin if required,
The dial will use the SIP and Webex. Join will try and use the Spark/Webex protocol to join.
Default is Dial
Define meeting settings as required, followed by either Save Only to run the test later or Save & Run to execute the test now.
Static Meeting Settings
Setting | Description |
---|---|
Name | Name of the test plan. |
Destination | sip URI or PhoneNumber. |
Force Existing Call Teardown | test_mate will end any active calls on the endpoints before executing the test plan. |
Mute Devices During Test | test_mate will signal to Mute the Microphone on the endpoint during the test. |
Pause Before MeetingID (secs) | The number of seconds test_mate waits before sending DTMF digits for the MeetingID after the call Connects. |
Pause Before PIN (secs) | The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects. |
Webex Account / Interface | Select the Webex account/interface used to control the endpoints. |
Endpoints Find Endpoints | Launches endpoint selector window. |
Endpoints Sync Webex Now | Synchronises endpoints for any new or removed endpoints. |
Endpoint Quality Monitoring | Enable or Disable Quality monitoring. |
Packet Statistics Wait (secs) | the number of seconds the call is left connected to allow statistics to be collected |
Test Content Sharing | TBC |
Schedule Test Run | TBC |
Scheduled Calendar invite Test Plan
This test plan simulates creating a scheduled meeting via a calendar invite and triggering the scheduling associated with the calendar user.
High-level test plan.
Create Meeting -> Check endpoints accepted -> Dial endpoints into meeting -> Check stats -> End calls -> After meeting cleanup and collection of cloud stats if available.
Scheduled Common Meeting Settings
Setting | Description |
---|---|
Name | Name of the test plan. |
Meeting Name | The subject of the meeting (displayed in the calendar invite) |
Meeting Method | Select the meeting integration to be used, e.g. Microsoft Teams with CVI. |
Meeting Location | Depending on the integration, the location field within the meeting invite may be required to have a specific format/value. |
Force Existing Call Teardown | test_mate will end any active calls on the endpoints before executing the test plan. |
Mute Devices During Test | test_mate will signal to Mute the Microphone on the endpoint during the test. |
Removing Meeting After Test | After the test plan has been completed, remove the calendar invite. This is useful to remove any room reservations and clean up the calendar. |
Meeting Create Then Start Pause (secs) | The number of seconds test_mate waits before creating the meeting invite when the test is executed. |
Max Waiting Time For Device Acceptance (secs) | The number of seconds test_mate for meeting rooms to accept the invite. |
Calendar Check Polling Interval For Acceptance (secs) | The number of seconds test_mate checks in each interval to accept the calendar invite. |
Max Waiting Time For Meeting Body Update (secs) | The number of seconds test_mate waits for the meeting to be updated with the SIP URI details required to make the video call. |
Polling Interval For Meeting Body Update (secs) | The number of seconds test_mate checks for the SIP URI details in each interval. |
Pause Before PIN (secs) | The number of seconds test_mate waits before sending DTMF digits for the Pin after the call Connects. |
Wait Before Loading Webex Meeting (secs) | The number of seconds test_mate waits before validating each endpoint has joined the meeting and being admitted (if supported by the meeting type) successfully. |
Webex XAPI Request Pause After Error (secs) | Test_mate waits for the number of seconds test_mate waits after receiving a Xapi error to an endpoint command. |
Webex XAPI Request Retries After Error | The maximum number of retries test_mate attempts to a Xapi command. |
Microsoft Graph Interface | Select the Microsoft graph application registration user. |
Calendar User | Selects the Calendar user to schedule meetings for testing. |
Webex Account / Interface | Select the Webex account/interface used to control the endpoints. |
Endpoints Find Endpoints | Launches endpoint selector window. |
Endpoints Sync Webex Now | Synchronises endpoints for any new or removed endpoints. |
Endpoint Quality Monitoring | Enable or Disable Quality monitoring. |
Packet Statistics Wait (secs) | the number of seconds the call is left connected to allow statistics to be collected |
Test Content Sharing | TBC |
Schedule Test Run | TBC |
Cisco CVI / Microsoft Teams meeting
Endpoints are available to make a call.
Apply specific meeting settings (e.g. mute endpoints before calling).
Creates an online meeting via the Microsoft Graph interface (simulates an end-user creating the meeting in outlook).
Endpoints have accepted or declined the meeting in their calendar.
Dials the CVI bridge number that is dynamically generated with the online meeting.
Joins all endpoints to the call and confirms they have connected.
Collects meeting participants and meeting qualities if available from the provider.
Disconnect the endpoints
Any other after meetings tasks (e.g. Delete meeting invite).
Setting | Description |
---|---|
Collect Attendance Reports | Collects meeting qualities if available from the provider. |
Wait After Meeting For Attendance Reports (secs) | Number of seconds test_mate waits before retrieving stats from the provider. |
Cisco Webex using @meet @webex
Endpoints are available to make a call.
Apply specific meeting settings (e.g. mute endpoints before calling).
Creates a regular meeting (e.g. without selecting teams) via the Microsoft Graph interface (simulates an end-user creating the meeting in outlook).
Endpoints have accepted or declined the meeting in their calendar.
Dials the Webex @webex or @meet details dynamically generated by Webex parsing the location field.
Joins all endpoints to the call and confirms they have connected.
Collects meeting participants and meeting qualities if available from the provider.
Disconnect the endpoints
Any other after meetings tasks (e.g. Delete meeting invite).
Setting | Description |
---|---|
Location @meet | @meet is a keyword that triggers a Webex integration within the calendaring service, which uses the Webex personal meeting room for the invite. |
Location @webex | @webex creates a one-time meeting with associated Webex space for the invite. |
Collect Webex Stats Report | Collects meeting qualities if available from the provider. |
Wait After Meeting For Webex Stats (secs) | Number of seconds test_mate waits before retrieving stats from the provider. |
refer to the following link for the Microsoft Webex location integration https://help.webex.com/en-us/article/nwtulzs/Webex-App-%7C-Schedule-a-meeting-using-@webex-or-@meet
Cloud Video Test Runs Results
Webex Edge for Devices - OnPrem/HCS registered
On-prem registered devices can be controlled via Webex Edge for Devices, allowing xApi commands to be proxied securely to the endpoint via Webex. It's also possible to provide enhanced/optimised Webex meetings to On-prem devices whilst retaining their existing call flows.
Refer to the following for further information.
https://help.webex.com/en-us/article/cy2l2z/Webex-Edge-for-Devices
Webex Call routing
Cloud Registered
Video calls will route within Webex as normal
Webex Call routing - Sip
Video calls will route via the existing signalling path, e.g. via an expressway to the internet to Webex.
This is the default setting with Webex Edge for Devices.
Webex Call routing - Webex optimised
Video calls to Webex will route directly from the Endpoint to the internet, e.g. not via expressways. This requires extra configuration to enable this feature and possible network/firewall changes. Non-Webex Video calls will route via the existing signalling path, e.g. via an expressway.
Troubleshooting
API errors
Problem: Blank page after submitting Webex credentials
Suggested course of action:
Check permissions for the integration account match in test_mate and Webex.
Try using an alternate browser such as Firefox.
Webex requires the request permission from test_mate to match the configured permissions in Webex. Else the integration will fail.
Problem: Expired credentials/tokens in Webex
Suggested course of action:
Click on Update / Set Credentials to reauthorise with the account that grated the Webex integration and/or check the correct details supplied.
Problem: Missing Redirect URL for Webex integration account
Suggested course of action :
Ensure the Redirect URL is the same as displayed within the test_mate interface.
Remove any spaces from the URL's beginning and/or end when using copy/paste to Webex.
Deleting and re-adding an interface will create a new unique redirect URL
Problem: Microsoft Graph ErrorAccessDenied
Suggested course of action :
Check the correct application account/client id used with the proper tenant id.
Check application account/client id has access to mailboxes/calendars selected for test plan(s).
Check API permissions in azure AD setup correctly and granted.
Your organisation may limit access to all mailboxes/calendars. Check with your Azure/o365 admin
https://docs.microsoft.com/en-us/graph/auth-limit-mailbox-access#configure-applicationaccesspolicy
Problem: Webex Device access error
Suggested course of action :
Check Webex admin permissions for the Webex end-user scheduling the meeting.
Check device is online and registered.
If your organisation won't allow admin read-only access to Webex, The integration user needs to be added to every workspace used for test_mate device testing.
Internet / Proxy Errors
Problem: Wrong Protocol
Suggested course of action:
Check the correct protocol has been specified, e.g. HTTP
Check the correct port number provided
Problem: Wrong User/Password
Suggested course of action:
Check correct username / password entered ( password could be case sensitive depending on the 3rd party proxy).
Check Proxy allows access to the domains required for API calls to the providers.