Skip to main content
LTTS

LTTS

Quick Links mini

  • Search
  • info@LTTS.com
  • English
  • 日本語
  • Israel
  • German
  • Contact
  • Engineering the change
  • Industry
    • Transportation
      • Aerospace
      • Automotive
      • Rail Transportation
      • Travel & Hospitality
      • Trucks & Off-Highway Vehicles
    • Industrial Products
      • Building Solutions
      • Lighting Engineering
      • Power electronics & drives
      • Renewable Energy
      • Test & Measurement
      • Power Generation & Transmission
    • Plant Engineering
      • CAPEX Project E/EPCM Services
      • Digital Engineering Services
      • Engineering Reapplication & Global Rollouts
      • Integrated Asset Management Services
      • Operational excellence
      • Plant sustenance & management
      • Regulatory compliance engineering
    • Semiconductors
      • IP Core Solutions
    • Media & Entertainment
      • Cable & Broadcasting
      • OTT
      • Rdk
      • Set-Top Boxes
      • Smart Home
    • Consumer Electronics
      • Consumer IoT
      • Enterprise Devices
      • Personal Devices
    • Healthcare
      • Healthcare Providers
      • Medical Devices
    • Telecommunications
      • 5G
      • SDN/NFV
      • Wireless Networks
      • Wireline Networks
    • Oil & Gas
      • Digitalization
      • Oil Field Equipment
      • Owners and Operators
    • Software Products
  • Services
    • Product Engineering
      • User Experience
      • Mechanical Design
        • CAE & CFD
        • CAx Automation
        • Integrated Design, Validation & Testing
      • Security
        • Security Monitoring
        • Security Solutions
        • Security Services
      • Software
        • Cloud Engineering
        • DevOps
        • Engineering Analytics
        • Immersive Experiences
        • Voice Innovations
      • Hardware
        • Embedded Systems
        • Wearables Engineering
        • Testing
        • Sustenance
      • VLSI
      • Testing & Validation
    • Manufacturing Engineering
      • PLM on Cloud
      • aspenONE on Cloud
      • Plant Design & Engineering
      • Digital Factory & Simulations
      • Line Expansion & Transfer
      • Frugal Product Design
      • Asset Care
      • Accelerated Operations
      • Manufacturing Automation
      • Agile Supply Chain
      • Content Engineering
    • Operations Engineering
      • Connected Service Support
      • Integrated Content Management
      • Intelligent Building Management
      • Sourcing & Procurement
    • Engineering Consultancy
      • Industry 4.0
      • Product Strategy
      • Security
      • Smart Factory
      • Sustainability
  • Solutions
    • FlyBoard®Advanced Digital Signage Solution
    • Connected Security Integrative Zero Trust Architecture
    • ESM Energy and Sustainability Manager
    • Cogmation Device Test Automation Framework
    • i-BEMSIntelligent Building Experience Management
    • UBIQWeise 2.0 Device to Cloud IoT Platform
    • AiKno™ Machine Learning, NLP & Vision Computing
    • Semiconductor IP For Security, Communication & Verification
    • nBOnnB-IoT Protocol Stack
    • Avertle®AI Predictive Maintenance Solution
    • ARC Asset Reliability Centre
    • CHEST-rAi™ AI Chest X-Ray Radiology Assist Suite
    • AnnotAi
    • AiCEArtificial Intelligence Clinical Evaluation
  • Insights
    • Blogs
      • 6 DevOps Trends to Watch Out for in 2023
      • A Validation Overview of ECU Communication Protocols in Automotive Audio Management
      • QOE Testing Paradigms for Media Streaming Platforms: Is Automation Your Answer
      • SIMD/SAMD: Everything You Need to Know
    • News
      • L&T Technology Services reports 21% growth and crosses the ₹300 crore mark in Net Profit in Q3FY23
      • LTTS Selected as Strategic Engineering Partner to Airbus for Advanced Capabilities and Digital Manufacturing Services
      • LTTS agrees to acquire Smart World & Communication business of L&T
      • Top global enterprises and engineers named winners of the inaugural Digital Engineering Awards
    • POV
      • Hidden Correlations Shaping the Future of European Enterprises
      • From the Super Bowl to 5G
      • Vehicle-to-Everything (V2X) Enabling Smarter, Safer, and Greener Transportation
      • Will 5G Technologies Drive a New Future for Media & Entertainment?
    • eBooks
      • The Art of Cyberwar
      • Digital Twin - The Future of Manufacturing
      • Digitalising Wind Energy Ecosystem
      • INDUSTRY 4.0: The Future Is Now
      • Digital Engineering Explained
      • Sustainability Engineering
  • Explore LTTS
    • About Us
    • Nearshore Centers
    • Testimonials
    • Events & Webinars
    • News & Media
    • Board of Directors
    • CSR
    • Accolades
    • Quality Management
    • Analysts
    • Careers
    • Investors
    • Media Kit
    • Resources
    • Alliances
    • Sustainability
  • Contact
 

Media & Entertainment

QoE Testing Paradigms for Media Streaming Platforms: Is Automation Your Answer

  1. Home
  2. Blogs
  3. Industry
  4. QoE Testing Paradigms for Media Streaming Platforms

QoE Testing Paradigms for Media Streaming Platforms: Is Automation Your Answer

 Quality of Experience (QoE) paradigms
Published on: 28 Oct, 2022
0 Views
0 comments
Share This Article:
  • Twitter
  • Facebook
  • Linked in
Media & Entertainment
Quality of Experience
OTT
QoE
Automated QoE
OTT Media
OTT Platforms

In a previous blog, I spoke about how evolving Quality of Experience (QoE) paradigms, especially in testing and validation, are transforming the Over the Top (OTT) media landscape. With features like on-demand and customized content availability, multi-device viewing capabilities, and superior quality helping drive, define and reshape the ecosystem, it is no surprise that the OTT revolution, powered by QoE, will only continue to get stronger in the days to come. 

Let us delve a little deeper in the details of QoE and its testing to understand its role and impact on the success of global media streaming platforms.

The Factors Affecting QoE

QoE is primarily impacted by four workflows :

  • Player Workflows: Where new application features are added or updated, new SDKs are integrated, etc.
  • Content Workflows: Where the mezzanine video get transcoded, encrypted , packaged, and pushed to the CDN
  • Encoding Profile Workflows: Where  there are multiple sub-changes and optimizations in the Transcoding section of content Workflow
  • Infra Workflows: Where other various other systems of the OTT application backend like DRM , protocols, watermarking, and CMS is updated

Quality of Experience (QoE) paradigms                                                                                                                      The QoE  Quality Workflow

These key components of the OTT ecosystem require extensive testing in the pre-production environment. And once the application is released (or pushed) to the production/customer stage, a player beacon captures and reports on the customer QoE, establishing a direct correlation between this vital indicator and the success of the streaming platform.

What we need to remember here is that since any QoE entity tested is a combination of various parts of the OTT ecosystem, it is important to test it systematically and in an organized manner - with a clear understanding of the viewer ecosystem and the expected usage patterns.

Manual vs. Automated QoE Testing: Navigating the Conundrum

End device testing is the best way to identify the QoE of a media streaming platform. A QoE test comes under the NFR category, which is undertaken in considerably lesser facilities as compared to functional tests.

Again, QoE testing today is mostly manual or semi-automated (at best), which results in the need for considerable time and investments to ensure and deliver on the expected parameters. With viewer demand patterns continuing to be significantly dynamic, there is an urgent need to optimize on these cycle times to ensure enhanced outcomes.

The scenario is further complicated by the fact that manual (or semi-automated) QoE testing does not always help uncover the right correlation required, including such vital aspects like start up time, buffering across bandwidth streams, ABR performance, and the impact of other infrastructural changes.

We therefore need to compare and contrast the changes and drive the development of new streaming optimization levers for redefining existing QoE testing paradigms.

Enter automation.

While the automation of QoE Testing is considerably more complex than normal functional automation due to kits need for an altogether different logic and thought process, it is not an unachievable target. The journey initially comprises of various functional automation scripts and logics when we automate an application and navigate through its EPG to the playback scenarios.

What is critical here is the need to design the APIs and scripts capable of the streaming QoE KPIs from video playback initiation. The requirement is to focus on the right usage of automation scripting techniques and capable scripting of APIs provided within the desired automation tool or framework.

We need to remember that the normal NFR and performance measurement APIs or techniques may not be directly usable in automated testing scenarios. And further, the KPIs have to be measured along with a set of other parameters that define the state of the system when the measurement had taken place. Measuring buffering ratio against varying Wi-Fi signal strength levels (received at the end device) is one example that comes to mind.

Our target is not only to rely on the return from the QoE APIs , but to aggregate and compare the right metadata and environmental conditions to derive meaningful insights where the presentation layer of these test and benchmark data becomes important. A good QoE testing system, therefore, not only provides the APIs or functions to measure the relevant KPIs, but is also capable of network manipulation, collecting network requests, application or player logs, and device logs etc.

Such an auto, mated solution should be capable of automatically searching for all data points necessary to triage and decide on actions to be taken for improvement. This capability is a clear line of separation between QoE tools built for purpose and NFR capable test automation tools.

Challenges in QoE Automation

  • Measuring the KPIs in 20 milli second intervals or with frame level accuracy
  • Power-intensive GPU-based based processing
  • Bringing together all relevant metadata to correlate and generate meaningful insights
  • Approach  to reliably manipulate network models
  • Availability of an applicable set of network model or templates to apply for QoE tests

Navigating the Future

The challenges and roadblocks aside, the future clearly belongs to a renewed focus on ensuring and delivering a best-in-class QoE, one that would help differentiate the target streaming platform from its rivals and competitors. And automated testing, powered by deep digital engineering domain expertise, is the way forward.

Quality of Experience (QoE) paradigms

Operation

1.A QoE test is started off the back of a CI trigger, schedule or manual execution

2.QoE Test Manager kicks off the test on a specific device

3.QoE Test solution automate the app into a play ready state

4.Cogmation is independent of the player, simulating what the customer would see

5.A common test pattern stresses the player across a range of likely scenarios

6.A network proxy controls the stream being viewed, the bandwidth its viewed at and records the data exchanged

7.Asset playback runs , containing a number of stop-start events and advertisements

8.The specially developed QoE metrics Capture API captures key facets of QoE

9.QoE KPIs captured are reported with relevant metadata

10.video analytics reported values are validated

11.Results are presented in a report and returned to the calling CI pipeline as a pass/fail where appropriate

Authors

Rakhil Kalarithottathil, Global Solution Architect, Media & Entertainment, LTTS
Rakhil Kalarithottathil
Global Solution Architect, Media & Entertainment, LTTS

Related Blogs

Rakhil Kalarithottathil
Transforming the OTT Game: QoE Validation in the Emerging Landscape
20 May, 2022
Ashish Moghe
Hyper-Personalization: The Holy Grail of OTT Success
19 Apr, 2022
Archit Shah
AI and OTT Media: How Artificial Intelligence is Shaping the Future of OTT Markets
28 Apr, 2022
Leave a Comment
About text formats

Comments

No Comments

×Explore
  • Industry
  • Spotlight
  • ×
  • Automotive
  • Consumer Electronics
  • Industrial Engineering
  • Lighting & Building Solutions
  • Media & Entertainment
  • Medical Devices
  • Oil & Gas
  • Plant Engineering
  • Power Electronics
  • Renewable Energy
  • Semiconductors
  • Industrial Products
  • Transportation
  • Telecommunications
  • 5G
  • Cloud engineering
  • Cyber security
  • Embedded systems
  • Industry 4.0
  • Smart Manufacturing
  • Smart Products
  • Sustainability
  • Artificial Intelligence
  • AR/VR
  • Image Processing
  • Connected Healthcare
  • Smart Factory
  • Digital Twins
  • Building Automation
  • Autonomous Transport
  • Robotics
  • Digital Entertainment
  • Machine Learning
  • UI/UX
  • Manufacturing Automation
  • Smart Sourcing
  • Simulation
  • Software Defined Networking
  • Telehealth
  • Wearables
  • Design Thinking
  • IoT Security
  • ER&D Hackathon 2019
  • Digital Media
  • The New Normal
  • Data Mesh
  • Media
  • parent-company-logo.png
  • Need Help
  • Contact Us
  •  

Contact Us

By clicking Submit, you agree to the Privacy Policy

  • Engineering the change
  • Industry
    • Transportation
    • Industrial Products
    • Plant Engineering
    • Semiconductors
    • Media & Entertainment
    • Consumer Electronics
    • Healthcare
    • Telecommunications
    • Oil & Gas
    • Software Products
  • Services
    • Products
      • CAE & CFD
      • CAx Automation
      • Cloud Engineering
      • DevOps
      • Embedded Systems
      • Engineering Analytics
      • Immersive Experiences
      • Integrated Design, Validation & Testing
      • Security Monitoring
      • Security Solutions
      • Security Services
      • Sustenance
      • Testing
      • Testing & Validation
      • User Experience
      • VLSI
      • Voice Innovations
      • Wearables Engineering
    • Manufacturing
      • PLM on Cloud
      • aspenONE on Cloud
      • Plant Design & Engineering
      • Digital Factory & Simulations
      • Line Expansion & Transfer
      • Frugal Product Design
      • Asset Care
      • Accelerated Operations
      • Manufacturing Automation
      • Agile Supply Chain
      • Content Engineering
    • Operations
      • Connected Service Support
      • Integrated Content Management
      • Intelligent Building Management
      • Sourcing & Procurement
    • Consultancy
      • Industry 4.0
      • Product Strategy
      • Security
      • Smart Factory
      • Sustainability
  • Solutions
    • i-BEMS
    • Connected Security
    • nBOn
    • UBIQWeise 2.0
    • ESM
    • AiKno™
    • Cogmation
    • Avertle®
    • ARC
    • Chest-rAi™
    • AiCE
  • Insights
    • Blogs
    • News
    • POV
    • eBooks
  • Explore LTTS
    • About Us
    • Nearshore Centers
    • Testimonials
    • Events & Webinars
    • News & Media
    • Board of Directors
    • CSR
    • Accolades
    • Alliances
    • Quality Management
    • Sustainability
  •  
  •  
  •  
  •  
  •  
^
  •  
  •  
  •  
  •  
  •  

© 2023 L&T Technology Services Limited. All Rights Reserved.

  • COPYRIGHT & TERMS
  • PRIVACY
  • Site Map
  • info@LTTS.com