Testing embedded systems and application using hardware-in-the-loop as a service (HILAAS)

Patent metadata

Field	Value
Publication number	US-10983897-B2
Application number	US-201815883290-A
Country	US
Kind code	B2
Filing date	Jan 30, 2018
Priority date	Jan 30, 2018
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

Embodiments for testing embedded systems and their applications in an Internet of Things (IoT) environment by a processor, denoted as a Hardware-in-the-Loop as a Service (HiLaaS). In a simulated environment, one or more simulated entities and one or more real entities in a networked system may be tested in real-time according to received control parameters, for a price. The price is estimated by the system, based on other parameters, and offered to the user to accept or reject. Alternatively, the user may specify the price, the system estimates control parameters, and the user can accept or reject the control parameters. One or more properties of the one or more entities, the network system, or combination thereof may be estimated based on the testing of the one or more simulated entities, when the price and control parameters are accepted.

First claim

The invention claimed is:

1. A method, by a processor, for testing networked systems in an Internet of Things (IoT) environment, comprising:
receiving one or more control parameters and information relating to a simulation environment;
testing, as a testing as a service (TaaS) in the simulation environment, one or more entities associated with a network system and one or more simulated entities associated with the network system according to the one or more control parameters, wherein a number of the one or more simulated entities under test is determined and adjusted in real-time according to test data received from the one or more entities comprising actual entities such that contextual and temporal information associated with a physical environment maintaining the one or more entities is used to modify the number of the one or more simulated entities under test, and wherein the number of the one or more simulated entities under test is different than a number of the one or more entities of the network system; and
estimating one or more properties of the one or more entities and the network system based on the testing of the one or more simulated entities such that test data from the one or more simulated entities is assimilated with the test data from the one or more entities associated with the network system to determine the estimation.

2. The method of claim 1 , wherein testing as the TaaS in the simulation environment further includes:
estimating a price for testing in the simulation environment according to the one or more control parameters; or
receiving an approval of the estimated price for the testing.

3. The method of claim 1 , wherein testing as the TaaS in the simulation environment further includes:
receiving a maximum threshold price, a range of pricing, or a combination thereof for testing in the simulation environment;
adjusting one or more control parameters according to the maximum threshold price, the range of pricing, or a combination thereof for testing in the simulation environment; or
receiving an approval of the adjusted control.

4. The method of claim 1 , wherein receiving the one or more control parameters and the information relating to the simulation environment includes:
receiving a description of the simulation environment;
receiving a description of the network system, wherein the description includes at least a topology of the network system;
receiving historical data related to operations of the network system;
receiving scheduling data for a selected time period of the operations of the network system;
receiving real-time streamed data from the one or more entities; or
receiving real-time streamed data from one or more third-parties.

5. The method of claim 1 , wherein receiving the one or more control parameters further includes:
receiving a selected time to perform the testing and provide testing results;
receiving one or more conditions for the testing the one or more simulated entities;
receiving one or more conditions for the testing the one or more entities of the system under test; or
receiving a measure of accuracy of the testing.

6. The method of claim 1 , further including prior to testing as the TaaS in the simulation environment:
estimating a number of the one or more entities to simulate over a selected period of time according to the one or more control parameters;
estimating a number of interactions between the one or more entities, the one or more simulated entities, and an environment associated with the networked system according to the one or more control parameters; or
estimating one or more computing systems needed to simulate the one or more entities and the number of interactions over a selected time period according to the one or more control parameters.

7. The method of claim 1 , wherein the testing further includes:
simulating the one or more entities according to the control parameters; and
updating the simulation environment in real-time during the simulating, wherein the updating includes at least historical data of operations of the network system.

8. The method of claim 1 , further including providing real-time streamed data for the one or more simulated entities as output of the testing of the one or more simulated entities.

9. The method of claim 1 , further including providing an interface for remote access by the one or more entities, a user, or a combination thereof for accessing the simulated environment.

10. The method of claim 1 , further including:
receiving a description of a simulation environment and price;
estimating the one or more control parameters of a test according to the price and the description of a simulation environment;
testing the one or more entities associated, the one or more simulated entities, or a combination thereof according to the control parameters upon acceptance of the one or more control parameters; and
estimating the one or more properties of the one or more entities, the network system, or combination thereof based on the testing of the one or more simulated entities.

11. A system for testing networked systems in an Internet of Things (IoT) network environment, comprising:
one or more computers with executable instructions that when executed cause the system to:
receive one or more control parameters and information relating to a simulation environment;
test, as a testing as a service (TaaS) in the simulation environment, one or more entities associated with a network system and one or more simulated entities associated with the network system according to the one or more control parameters, wherein a number of the one or more simulated entities under test is determined and adjusted in real-time according to test data received from the one or more entities comprising actual entities such that contextual and temporal information associated with a physical environment maintaining the one or more entities is used to modify the number of the one or more simulated entities under test, and wherein the number of the one or more simulated entities under test is different than a number of the one or more entities of the network system; and
estimate one or more properties of the one or more entities and the network system based on the testing of the one or more simulated entities such that test data from the one or more simulated entities is assimilated with the test data from the one or more entities associated with the network system to determine the estimation.

12. The system of claim 11 , wherein the executable instructions that when executed cause the system to:
estimate a price for testing in the simulation environment according to the one or more control parameters;
receive an approval of the estimated price for the testing;
receive a maximum threshold price, a range of pricing, or a combination thereof for testing in the simulation environment;
adjust one or more control parameters according to the maximum threshold price, the range of pricing, or a combination thereof for testing in the simulation environment; or
receive an approval of the adjusted control.

13. The system of claim 11 , wherein, pursuant to receiving the one or more control parameters and the information relating to the simulation environment, the executable instructions that when executed cause the system to:
receive a description of the simulation environment;
receive a description of the network system, wherein the description includes at least a topology of the network system;
receive historical data related to operations of the network system;
receive scheduling data for a selected time period of the operations of the networ

Assignees

• IBM

Inventors

• Epperlein Jonathan P
• Gambella Claudio
• Griggs Wynita M
• LASSOUED Yassine
• Marecek Jakub
• Mevissen Martin
• Monteil Julien
• Ordonez-Hurtado Rodrigo H
• Russo Giovanni
• Shorten Robert
• Zhuk Sergiy

Classifications

• H04L41/12

  Discovery or management of network topologies · CPC title

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• H04L43/20

  the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV · CPC title

• H04L41/40 Primary

  using virtualisation of network functions or resources, e.g. SDN or NFV entities · CPC title

• H04L43/50

  Testing arrangements · CPC title