Polish Telemetry Project – presentation of the course and results of the research and development works and research experiments
Within the framework of the Polish Telemetry Project, a conference was held at the Warsaw School of Economics on 26 February 2020 which presented the tests of measuring devices performed by the National Institute of Telecommunications – State Research Institute and also the research experiments carried out by the Institute of Statistics and Demography of the Warsaw School of Economics.
The first part of the conference saw the presentation of the results of the laboratory tests on the measuring devices carried out by the National Institute of Telecommunications – State Research Institute. The methodology of the research, its procedure and conclusions were discussed.
The complexity of this issue lies in the fact that the measurement should be performed as accurately as possible. One problem is the selection and distribution of sensors in the homes of the panellists, a second is the software for those devices, as that determines the quality of the recognition of the sound samples, and a third problem is the ICT system used to process the data. “All of this had to be examined,” said Jerzy Żurek, Director of the National Institute of Telecommunications – State Research Institute.
5.9 million observations were carried out, which took a total of 1,174 hours of testing time. There were 26 devices from six sensor suppliers tested, which represents 80% of all such devices available on the Polish market. 11 devices (4 types) scored below average, while 15 devices (5 types) scored above average.
“Accurate recognition was achieved after 12 seconds – which is five times better than the present methods. Currently, advertising blocks are examined on a minute-by-minute basis, and each spot is an average rather than an exact result. Our research will make it easier to manage content and budgets. This is of great importance for the entire market,” said Andrzej Garapich, an expert for the Polish Telemetry Project.
The main conclusions of the tests were:
• There are significant differences in efficiency between each of the devices
• Interference has no critical impact on recognition
• The programme type has little impact on the efficiency of recognition
• Keeping the mobile meter in a bag or backpack affects the efficiency of measurement
• The programme category has little impact on its recognition
• Differences in the level of recognition for radio and television programmes are small
• Personal sensors are able to handle single interferences relatively well
• Personal telemetry sensors may be used for testing media consumption in various everyday conditions
The second part of the conference presented the methodology, results and conclusions of the research experiments carried out jointly with the Institute of Statistics and Demography from the Warsaw School of Economics. The experiments examined the ways and frequency of use of electronic media depending on socio-demographic characteristics, and also tested the possibilities of recruiting households and their members onto the telemetry panel. The research tools were verified, and the organisational and technical aspects of the research were checked. Households and their members over 4 years of age were surveyed, including people aged 16 and older who are studying away from home.
“One of the key conclusions is the confirmed flexibility of the shape of the research panel and the initial survey, which will allow a high quality of measurement across all age groups through the selection of both individuals and also entire households for the survey, allowing the expectations of broadcasters and marketers alike to be met,” said Andrzej Olszewski, an expert from the Polish Telemetry Project team.
Despite the popularity of the internet, television is still the most important medium when it comes to receiving local, national and global information. The simultaneous use of several devices at once has become more common – for example, using smartphones or tablets while watching TV or listening to the radio. 30% of people aged 16+ use several devices at once in their consumption of electronic media. The most common activities performed alongside other activities are surfing the internet, watching TV or videos, and talking on the phone.
The experiments involved carrying out 1,005 household interviews and 1,984 individual interviews. This is a very large number, which augurs well for building the new research panel quickly.
“The plan is that by the end of the second quarter the position of all parties should be established and decisions taken which will allow the assumptions to be prepared for the technology tenders and recruitment of the panel. If that happens, we will probably have a working prototype in the first half of 2021, and by the end of next year we want to have the panel,” Andrzej Olszewski added.
As a result of the experiments, preliminary recommendations were presented for the initial survey:
• there should be a combination of individual interviews and household interviews
• the Initial Survey should be conducted by means of a representative method, based on a random sample
• due to the very high planned number of households participating in the survey, a much more detailed stratification of the first-stage random sampling units (statistical districts) is recommended
• the Initial Survey, just like the pilot survey, should be carried out using the CAPI technique, i.e. face-to-face interviews with an interviewer, and the use of a notebook to record the responses provided by the respondents
• the Initial Survey should be performed using the two-stage stratified sampling method to draw the base sample and the reserve samples
• use of a corresponding integrated weighting system as in the pilot survey
• linking the reward system for interviewers with the required effectiveness of the Initial Survey by the company/companies carrying it out
“Today we are one of the world’s most advanced teams working on devising a market-viable solution for measuring media consumption – based on single-source measurement. The next step will be to build a prototype – ready to be deployed as a target solution. This will serve as the basis for developing a solution which will be an independent currency, but one accepted and recognised by the market. We are counting on the results of the work presented being seen as an invitation for further discussion on a new standard for measuring media consumption. At the same time, we repeat our invitation to work further with our expert group, within the proposed methodological framework, on our project,” said Michał Wigurski, Director of the Polish Telemetry Project.
The Polish Telemetry Project envisages a telemetry panel with 26,000 respondents as a single-source measurement of TV, radio and the internet – measuring media consumption outside the home and on mobile devices. In addition, the project will launch an RPD panel with a target base of 11 million households. An important aspect is the already-mentioned single source of data, which means that the data on the various media – television, radio and the internet – will come from the same panellist.