Innovations for Outpatient Care: HSBI Researchers Are Equipping the KogniHome in Bielefeld with Sensors and AI Technology.

Bielefeld (hsbi). SHARLY notices pretty much everything: when the fridge is opened, when someone enters a room, when the light is on in the bedroom. By tracking the home’s water usage, the system even knows if the shower is being used or the toilet is being flushed. Thanks to SHARLY, which stands for “Smart Home Agent Really”, the KogniHome in Bielefeld is now somewhat reminiscent of “Big Brother”.

But fear not:  the academics who are working scientifically on this sensor-laden research apartment in the Bethel area are not interested in indiscretions. “What’s important to us is to find out how artificial intelligence and a networked home environment can help to ensure that people with physical or mental disabilities can live independently and safely as long as possible within their own four walls,” says Prof. Dr. Thorsten Jungeblut.

Privacy is retained but the level of patient wellbeing – good, medium or poor – is made evident

The professor of the Industrial Internet of Things at HSBI dreams that soon, the use of data that has been evaluated by an intelligent system will enable relatives and nursing service providers to be kept up to date remotely, so that they can know whether the care recipient is doing well or whether action is required. “Imagine if one day there was simply an app that could let you know using a simple traffic light system whether everything is okay with your elderly parents, or whether action is required in the medium term or immediately,” says Jungeblut, casting the vision for his work. “Wouldn’t that be useful?”

The professor and his doctoral candidate Justin Baudisch also have a few solutions up their sleeves regarding the critical “Big Brother” factor. “Using an evaluation system that simply categorises the situation as good, medium or poor goes a long way to protecting the resident’s privacy,” explains Baudisch. “And we are working with sensors, not cameras. The collection and analysis of the sensor data takes place locally.” Baudisch affirms that if data does ever leave the protected environment – for example, in order to aid research endeavours – it is homomorphically encrypted. “This is an innovative procedure whereby the unique substantial content of the data does remain but it is not possible to allocate it to a specific person.”

Affordable minimal sensor technology makes system implementation more feasible

SHARLY is a powerful software environment that Jungeblut and Baudisch are optimising in a step-by-step process. On a monitor in the KogniHome, the system schematically and very discreetly visualises what is currently happening in the apartment. A test subject tries it out by lying on the bathroom floor. Two blue squares appear on the screen and show the location where the simulated fall took place. The bathroom window is open – another fact that is reflected on the monitor. If nothing about the situation changes for a certain duration, the system could trigger an alarm in order to call emergency services to the scene.

Of course SHARLY’s passion for collecting data isn’t only helpful in an emergency situation.  Since the system saves and evaluates a huge quantity of sensor data, over time it establishes an overall picture of typical, normal routines. As soon as variations from this norm arise, it can provide useful indications, which caregivers can then investigate. The information analysed includes sensor data from motion detectors, light switches, doors, hatches and windows as well as from smart, networked household devices such as coffee machines, robot vacuum cleaners and scales. SHARLY also processes everything that smart meters monitor, i.e. heating, electricity and water.

“An advantage of our approach is that these micro sensors are now used as standard when fitting out buildings and by bathroom and kitchen manufacturers,” says Jungeblut. “The technology is therefore relatively cheap, which means there is a good chance that in the medium term, our system could be used in the context of home-based care, for example.”Together with “Ambulante Geriatrische Rehabilitation Bielefeld GmbH” (a local company specialising in outpatient geriatric rehabilitation), the team plans to implement smart home sensor technology in the context of home-based rehabilitation. Furthermore, at a PVM GmbH site in Brackwede, there is an exhibition of the technologies used in KogniHome, where anyone can look at and try out the sensor-based technology. Baudisch and his doctoral supervisor have also developed an interface for a care management software package and are beginning to enter the testing phase for this, too.

Once data is homomorphically encrypted, AI training commences using HSBI’s “yourAI” infrastructure

But does the system really need this much data? Why does it even record whether the kitchen cupboard has just been opened and whether the coffee machine is in use? “It’s quite simple,” explains Justin Baudisch. “That’s the only way we can find out the patterns of behaviour that are normal – and therefore noncritical – and those that represent a divergence, which could point to a potential problem.” Yet for this analysis to be reliable, the software needs to do a great deal of learning and gradually get smarter. This is where AI comes in: As previously mentioned, to start with the data from the networked apartment is homomorphically encrypted. It is then transferred to a network of computers at HSBI called yourAI which has the capacity to process large quantities of data and train AI systems.

The training involves recording activity at the apartment and, on the basis of events that happen one after another, depicting sequences of action. These are mapped out in a graph structure. Over time, once more and more new sequences have been depicted, it becomes possible for deviations from this structure – such as sequences that have not happened before or slight changes in known sequences – to be recognised as anomalies.  “When there are anomalies, which is what we call deviations from the usual behaviour that has been learned in the past, contact is made with caregiving relatives or staff – or, if necessary, the emergency services – who can then respond as appropriate,” says Baudisch. If the AI system had not undergone enough training, a false alarm would be fairly likely. For this reason, the team is currently in the process of teaching the AI system the correct way to handle trends. For example, the system must take into account seasonalities such as weekends and the different seasons of the year.

Working in the context of demographic change and the labour shortage in the care sector

So collecting and analysing large volumes of data and using AI for data classification improves the reliability of the system. According to Prof. Jungeblut, however, there is another advantage to large volumes of data: “Monitoring activity over a long period of time can also make it easier to recognise and diagnose neurological conditions such as dementia.”Depression is another condition that can be indicated by changing activity patterns in the home environment. Or if the system determines that the volume of water used by the toilet has been constantly declining, this could indicate that the care recipient is gradually becoming dehydrated – a dangerous development that is particularly common among elderly people.

Against the backdrop of the ageing population, the labour shortage in the care sector and the extent to which relatives are overwhelmed by the nature and volume of their work, Prof. Jungeblut believes that enabling elderly people to live independently at home is an important goal of the healthcare system. This belief motivates his research and that of his doctoral candidate. The KogniHome is an excellent fit for their research work. The research apartment in Bethel, an area in Bielefeld’s Gadderbaum district, was established in 2014 as a collaborative project with 14 partners. Germany’s Federal Ministry of Education and Research provided eight million euros of funding to establish a future-proof model apartment that sought to make use of technological assistance systems in order to enable people with disabilities to live independently. The home is now thoroughly networked and equipped with all kinds of technological features, such as an innovative entrance door that can be opened by the emergency services using a QR code issued by SHARLY. The project is operated by an organisation made up of key figures in the healthcare, commercial and higher education sectors.

Those representing the higher education sector have particularly ambitious goals for the project. For example, in order to address data protection challenges even more vigorously and robustly in the future, HSBI’s Professor Jungeblut has a medium-term goal for data processing: Instead of data being collected locally, anonymised and then transmitted for analysis, data should instead be processed locally by the AI system. Yet due to limited processing capacity, AI-based pre-processing locally at the flat may necessitate additional steps. It would call for what is known as a “co-design” of the local hardware. “We are currently trying to achieve this. Our first step is to reduce the complexity of the model, for example by using quantisation approximation. This simplification reduces the demand for processing power without adversely affecting the precision of the AI model,” explains Jungeblut. So the KogniHome apartment is still an exciting place to be – and “Big Brother” remains discreet, continuing to be mindful of protecting resident privacy. (lk)

For more photo material, please contact presse@hsbi.de.