Skip to main content
Home

Search

Header (Main)
Industry
Mobility Mobility
Automotive
Vehicle Electrification

Vehicle Electrification

Explore
Aerospace Engineering
Optimized Aircraft Downtime and Overhead Costs for a Leading Global Airline with Automated Analytics

Optimized Aircraft Downtime and Overhead Costs for a Leading Global Airline with Automated Analytics

Explore
Rail Transportation
Railway Track Inspection System

Railway Track Inspection System

Explore
Trucks & Off-Highway Vehicles
On the Road to Connected Mobility: An ER&D Perspective

On the Road to Connected Mobility: An ER&D Perspective

Explore
On the Road to Connected Mobility

On the Road to Connected Mobility: An ER&D Perspective

Explore
Sustainability Sustainability
Discrete Manufacturing & Industrial Products
Building Technology & Smart Infrastructure
Electrical Power and Controls
Industrial Machinery
Unlock the future of manufacturing with Factories of the Future

Unlock the future of manufacturing with Factories of the Future

Explore
Process Manufacturing
Oil & Gas
Chemicals
FMCG
ISG Provider Lens™ 2024: Oil and Gas Industry Services and Solutions

ISG Provider Lens™ 2024: Oil and Gas Industry Services and Solutions

Explore
L&T Technology Services and bp sign multi-year engineering services partnership

L&T Technology Services and bp sign multi-year engineering services partnership

Explore
Tech Tech
HiTech
Consumer Electronics
Media & Entertainment
NexGen Comms
Semiconductors
L&T Technology Services Secures $50 Million Deal as a Strategic Partner from a global Network Provider

L&T Technology Services Secures $50 Million Deal as a Strategic Partner from a global Network Provider

Explore
MedTech
Revolutionizing Endoscopy with Software-Defined Innovation in Collaboration with NVIDIA

Revolutionizing Endoscopy with Software-Defined Innovation in Collaboration with NVIDIA

Explore
Public Infrastructure & Smart Cities
Integrated Smart Surveillance Project

Integrated Smart Surveillance Project

Explore
Software & Platforms
LTTS & SymphonyAI to provide AI-based transformation

LTTS & SymphonyAI to provide AI-based transformation

Explore
Services
Digital Engineering & Consulting Digital Engineering & Consulting
Artificial Intelligence
Cybersecure
Security Monitoring
Security Services
Security Solutions
Immersive Experiences
Industry 4.0
Product Consulting
Sustainability Engineering
Sustainable Smart World
5G
LTTS Completes Acquisition of Intelliswift

LTTS Completes Acquisition of Intelliswift

Explore
Product Engineering Product Engineering
Software Engineering
Cloud Engineering
DevOps
Engineering Analytics
Immersive Experiences
Sustenance & Maintenance
User Experience
Voice Innovations
Embedded Engineering
Embedded Systems
Sustenance
VLSI
Wearables Engineering
Mechanical Design
CAE & CFD
CAx Automation
Testing & Validation
Integrated Design, Validation & Testing
Lab as a Service
Testing
Enabling a Paradigm Shift in Testing An LTTS AI Perspective

Enabling a Paradigm Shift in Testing An LTTS AI Perspective

Explore
Manufacturing Engineering Manufacturing Engineering
Smart Manufacturing
Accelerated Operations
Digital Factory & Simulations
Plant Design & Engineering
Supply Chain Engineering
Sourcing & Procurement
Manufacturing & Planning
Accelerated Operations
Digital Factory & Simulations
Line Expansion & Transfer
Manufacturing Automation
New Product Development
Plant Design & Engineering
PLM on Cloud
Manufacturing Execution
Agile Supply Chain
Content Engineering
Material & Parts Management
Sourcing & Procurement
Asset Reliability Centre

Asset Reliability Centre

Explore
Plant Engineering Plant Engineering
CAPEX Project E/EPCM Services
Operational Excellence
Plant Sustenance & Management
Material & Parts Management
Regulatory Compliance Engineering
ISG Provider Lens™ 2024: Oil and Gas Industry Services and Solutions

ISG Provider Lens™ 2024: Oil and Gas Industry Services and Solutions

Explore
Solutions
AiCE
AiKno®
AnnotAI
ARC
Asset Health Framework
CHEST-rAi™
Connected Security
EDGYneer
ESM
EvQUAL
FlyBoard®
Fusion
i-BEMS
Nliten
nBOn
PSM
SafeX
Semiconductor IP
Sensor & Gateway Solution
UBIQWeise 2.0
Insights
Analyst Reports
Blogs
Brochures
Case Studies
eBooks
Events
PoVs
Webinars
Whitepapers
Careers
About Us
Accolades
Alliances
Analysts
Board of Directors
CSR
Engineer At Heart
Engineering The Change
Investors
Nearshore Centers
News & Media
Quality Management
Corporate Sustainability
Testimonials
Contact
Header (Secondary)
Search
Mail
Country
EN
DE
JP
HE
Contact

Breadcrumb

  1. Blogs
  2. Industry
  3. QoE Testing Paradigms for Media Streaming Platforms

QoE Testing Paradigms for Media Streaming Platforms: Is Automation Your Answer

Rakhil Kalarithottathil, Global Solution Architect, Media & Entertainment, LTTS
Rakhil Kalarithottathil

Global Solution Architect, Media & Entertainment, LTTS

Media & Entertainment

Published on 28 Oct 2022

min read

 Quality of Experience (QoE) paradigms

In a previous blog, I spoke about how evolving Quality of Experience (QoE) paradigms, especially in testing and validation, are transforming the Over the Top (OTT) media landscape. With features like on-demand and customized content availability, multi-device viewing capabilities, and superior quality helping drive, define and reshape the ecosystem, it is no surprise that the OTT revolution, powered by QoE, will only continue to get stronger in the days to come. 

Let us delve a little deeper in the details of QoE and its testing to understand its role and impact on the success of global media streaming platforms.

The Factors Affecting QoE

QoE is primarily impacted by four workflows :

  • Player Workflows: Where new application features are added or updated, new SDKs are integrated, etc.
  • Content Workflows: Where the mezzanine video get transcoded, encrypted , packaged, and pushed to the CDN
  • Encoding Profile Workflows: Where  there are multiple sub-changes and optimizations in the Transcoding section of content Workflow
  • Infra Workflows: Where other various other systems of the OTT application backend like DRM , protocols, watermarking, and CMS is updated

Quality of Experience (QoE) paradigms                                                                                                                      The QoE  Quality Workflow

These key components of the OTT ecosystem require extensive testing in the pre-production environment. And once the application is released (or pushed) to the production/customer stage, a player beacon captures and reports on the customer QoE, establishing a direct correlation between this vital indicator and the success of the streaming platform.

What we need to remember here is that since any QoE entity tested is a combination of various parts of the OTT ecosystem, it is important to test it systematically and in an organized manner - with a clear understanding of the viewer ecosystem and the expected usage patterns.

Manual vs. Automated QoE Testing: Navigating the Conundrum

End device testing is the best way to identify the QoE of a media streaming platform. A QoE test comes under the NFR category, which is undertaken in considerably lesser facilities as compared to functional tests.

Again, QoE testing today is mostly manual or semi-automated (at best), which results in the need for considerable time and investments to ensure and deliver on the expected parameters. With viewer demand patterns continuing to be significantly dynamic, there is an urgent need to optimize on these cycle times to ensure enhanced outcomes.

The scenario is further complicated by the fact that manual (or semi-automated) QoE testing does not always help uncover the right correlation required, including such vital aspects like start up time, buffering across bandwidth streams, ABR performance, and the impact of other infrastructural changes.

We therefore need to compare and contrast the changes and drive the development of new streaming optimization levers for redefining existing QoE testing paradigms.

Enter automation.

While the automation of QoE Testing is considerably more complex than normal functional automation due to kits need for an altogether different logic and thought process, it is not an unachievable target. The journey initially comprises of various functional automation scripts and logics when we automate an application and navigate through its EPG to the playback scenarios.

What is critical here is the need to design the APIs and scripts capable of the streaming QoE KPIs from video playback initiation. The requirement is to focus on the right usage of automation scripting techniques and capable scripting of APIs provided within the desired automation tool or framework.

We need to remember that the normal NFR and performance measurement APIs or techniques may not be directly usable in automated testing scenarios. And further, the KPIs have to be measured along with a set of other parameters that define the state of the system when the measurement had taken place. Measuring buffering ratio against varying Wi-Fi signal strength levels (received at the end device) is one example that comes to mind.

Our target is not only to rely on the return from the QoE APIs , but to aggregate and compare the right metadata and environmental conditions to derive meaningful insights where the presentation layer of these test and benchmark data becomes important. A good QoE testing system, therefore, not only provides the APIs or functions to measure the relevant KPIs, but is also capable of network manipulation, collecting network requests, application or player logs, and device logs etc.

Such an auto, mated solution should be capable of automatically searching for all data points necessary to triage and decide on actions to be taken for improvement. This capability is a clear line of separation between QoE tools built for purpose and NFR capable test automation tools.

Challenges in QoE Automation
  • Measuring the KPIs in 20 milli second intervals or with frame level accuracy
  • Power-intensive GPU-based based processing
  • Bringing together all relevant metadata to correlate and generate meaningful insights
  • Approach  to reliably manipulate network models
  • Availability of an applicable set of network model or templates to apply for QoE tests
Navigating the Future

The challenges and roadblocks aside, the future clearly belongs to a renewed focus on ensuring and delivering a best-in-class QoE, one that would help differentiate the target streaming platform from its rivals and competitors. And automated testing, powered by deep digital engineering domain expertise, is the way forward.

Quality of Experience (QoE) paradigms

Operation

1.A QoE test is started off the back of a CI trigger, schedule or manual execution

2.QoE Test Manager kicks off the test on a specific device

3.QoE Test solution automate the app into a play ready state

4.Cogmation is independent of the player, simulating what the customer would see

5.A common test pattern stresses the player across a range of likely scenarios

6.A network proxy controls the stream being viewed, the bandwidth its viewed at and records the data exchanged

7.Asset playback runs , containing a number of stop-start events and advertisements

8.The specially developed QoE metrics Capture API captures key facets of QoE

9.QoE KPIs captured are reported with relevant metadata

10.video analytics reported values are validated

11.Results are presented in a report and returned to the calling CI pipeline as a pass/fail where appropriate

Relevant Blogs

Transforming the OTT Game: QoE Validation in the Emerging Landscape
Hyper-Personalization: The Holy Grail of OTT Success
AI and OTT Media: How Artificial Intelligence is Shaping the Future of OTT Markets
Explore All

Stay Relevant With Us

Subscribe to our blogs

Rakhil Kalarithottathil, Global Solution Architect, Media & Entertainment, LTTS
Rakhil Kalarithottathil

Global Solution Architect, Media & Entertainment, LTTS

Rakhil Kalarithottathil is a leading expert in Media Broadcast & OTT technology, Quality Engineering, Media and Video Streaming and is currently a Global Solution Architect at LTTS. He has over a decade of experience specifically in the Media and Entertainment Industry. During this time, Rakhil handled Product Development, Technical Program Management, Technical Consulting, and Solutions for diverse customer segments in Media & New Media Domain.

Footer Navigation
  • Industry
    • Mobility
      • Aerospace Engineering
      • Automotive
      • Rail Transportation
      • Trucks & Off-Highway Vehicles
    • Sustainability
      • Discrete Manufacturing & Industrial Products
      • Process Manufacturing
    • Tech
      • Consumer Electronics
      • MedTech
      • Media & Entertainment
      • NexGen Comms
      • Semiconductors
      • Software & Platforms
      • Public Infrastructure & Smart Cities
  • Services
    • Digital Engineering
      • Artificial Intelligence
      • Cybersecure
      • Security Monitoring
      • Security Solutions
      • Security Services
      • Immersive Experiences
      • Industry 4.0
      • Product Consulting
      • Sustainability Engineering
      • Sustainable Smart World
      • 5G
    • Product Engineering
      • CAE & CFD
      • CAx Automation
      • Software Engineering
      • Cloud Engineering
      • DevOps
      • Embedded Systems
      • Engineering Analytics
      • Integrated Design, Validation & Testing
      • Lab as a Service
      • Sustenance
      • Testing
      • Testing & Validation
      • User Experience
      • VLSI
      • Voice Innovations
      • Wearables Engineering
    • Manufacturing Engineering
      • Accelerated Operations
      • Agile Supply Chain
      • Content Engineering
      • Digital Factory & Simulations
      • Line Expansion & Transfer
      • Manufacturing Automation
      • New Product Development
      • PLM on Cloud
      • Plant Design & Engineering
      • Sourcing & Procurement
    • Plant Engineering
      • CAPEX Project E/EPCM Services
      • Material & Parts Management
      • Operational Excellence
      • Plant Sustenance & Management
      • Sourcing & Procurement
      • Regulatory Compliance Engineering
  • Engineering The Change
  • Careers
  • Engineer at Heart
  • Resources
  • Solutions
    • AiCE
    • AiKno®
    • AnnotAI
    • ARC
    • Asset Health Framework
    • CHEST-rAi™
    • Connected Security
    • EDGYneer
    • ESM
    • EvQUAL
    • FlyBoard®
    • Fusion
    • i-BEMS
    • Nliten
    • nBOn
    • PSM
    • SafeX
    • Semiconductor IP
    • Sensor & Gateway Solution
    • UBIQWeise 2.0
  • About Us
    • Accolades
    • Alliances
    • Blogs
    • Board of Directors
    • Careers
    • CSR
    • Events & Webinars
    • Investors
    • Media Kit
    • Nearshore Centers
    • News & Media
    • Quality Management
    • Resources
    • Corporate Sustainability
    • Testimonials
LTTS
  •  Twitter
  •  LinkedIn
  •  YouTube
  •  Facebook
  •  Instagram
  • Copyright & Terms
  • Privacy
  • Sitemap
  • info@ltts.com

© 2025 L&T Technology Services Limited. All Rights Reserved.