INFORMATION ABOUT PROJECT,
SUPPORTED BY RUSSIAN SCIENCE FOUNDATION

The information is prepared on the basis of data from the information-analytical system RSF, informative part is represented in the author's edition. All rights belong to the authors, the use or reprinting of materials is permitted only with the prior consent of the authors.

 

COMMON PART


Project Number17-72-30003

Project titlePhysical basis of self-learning adaptive intelligent systems and their applications in biomorphic and anthropomorphic robotics

Project LeadHramov Alexander

AffiliationAutonomous noncommercial organization of higher education "Innopolis University",

Implementation period 2017 - 2020 

Research area 02 - PHYSICS AND SPACE SCIENCES, 02-402 - Nonlinear oscillations and waves

KeywordsNonlinear dynamics, theory of oscillations, dynamical chaos, anthropomorphic robotics, adaptive control system, artificial neuronal networks, complex networks, synchronization, machine learning, neuroscience, experiment, exoskeleton, big data


 

PROJECT CONTENT


Annotation
The proposed project is aimed for solving an actual scientific problem that lies at the intersection of physics, nonlinear dynamics, neuroscience, the creation of intelligent systems and robotics, related to the identification and studying of physical processes that determine the ability of living organisms to adaptive controlling motor systems in a complex and constantly changing environment for building strategies and specific methods of the intelligent control of biomorphic and anthropomorphic robots. The specific task of the project is the development, experimental verification and practical implementing advanced technologies of adaptive intelligent control of biomorphic and anthropomorphic robotic systems based on methods and approaches of nonlinear dynamics under conditions of dynamically changing environment. The topicality of this particular task is due to the need of increasing the level of automating the behavior of biomorphic and anthropomorphic robots [Waldron K.J., Tokhi M.O., Virk G.V. Nature-Inspired Mobile Robotics, World Scientific, 2013], their adaptation to an external dynamically changing environment, which will increase efficiency and expand the possibilities of their use in the social sphere, science and technology, including industrial production [Klimchik A. Efficiency evaluation of robots in machining applications using industrial performance measure // Robotics and Computer-Integrated Manufacturing 48. 12 (2017)], rescue operations [Hodson H. 2015 preview: Rescue robots go head-to-head // New Scientist 3000 (2014)], the elimination of the consequences of natural disasters [Nagatani, K. et al. / / J. Field Robot. 30, 44-63 (2013)], health care [Guang-Zhong Yang et al Medical robotics—Regulatory, ethical, and legal considerations for increasing levels of autonomy // Science Robotics 2. 4 (2017)], transport functions [Mataric M.J. Socially assistive robotics: Human augmentation versus automation // Science Robotics 2. 4 (2017)], space exploration [Barfoot T.D., Wettergreen D. Editorial: Special Issue on Space Robotics // J. Field Robot. 33. 2 (2016); Rus D., Tolley M. Design, fabrication and control of soft robots // Nature 521. 467 (2015)], service functions [Sprenger M., Mettler T. Service Robots // Business & Information Systems Engineering, 57. 271 (2015)], which corresponds to both the world trends in the use of anthropomorphic robotics, and contributing to the formation scientific and technological reserves, which ensure the economic growth of the Russian Federation through the development of new technologies and markets. The task is large-scale, since its solution involves the use of a complex approach that involves both fundamental research, including the development of theoretical foundations for the creation of intelligent self-learning control systems, based on numerical analysis and experimental radiophysical study of model neuron-like systems and modeling processes that provide human locomotor functions, Carrying out neurophysiological experiments and analyzing the signals of the electron Cephalogram and human myograms in order to identify the processes responsible for human interaction with surrounding objects and the realization of motor locomotor activity, and build on the basis of the obtained information an intelligent control system based on the methods and approaches of artificial intelligence, its training and testing on the example of a specific anthropomorphic robotic platform (Anthropomorphic robot AR-600 produced by SPA "Android Technics"). In the proposed project, three major areas of research can be identified, subjugated to the overall goals of the project, formulated above, and combining a number of interrelated and complementary tasks. Direction 1. "Developing theoretical bases of building nonlinear dynamic models, self-learning adaptive intelligent systems." The work in this direction implies carrying out a numerical analysis of the dynamics of networks of neuron-like elements with different configurations of the inter-element links in order to reveal the laws responsible for the formation of space-time structures, their evolution, switching between different types of dynamics under the effect of external influences. It is expected that the revealed regularities in the dynamics of model networks will be used in the design of intelligent self-learning automatic control systems. Direction 2. "Analysis and identification of the common mechanisms of evolution of neural connections, and switching between the different patterns of neural and muscle activity of the person, providing adaptive motor activity and spatial orientation when controlling anthropomorphic manipulators." The work in this direction is experimental and will be performed on the unique equipment that is planned to be acquired from the NGO "Android Technology" - the leading manufacturer of anthropomorphic robotic systems in Russia. As a result of the work, a database of electroencephalograms and myograms along with signals containing commands for direct control of the robot will be created. This base will be used to train intelligent control systems. Direction 3 "Creating intelligent system of controlling biomorphic robotic devices with the ability of automatic adaptation to changing environmental conditions" Within the framework of this direction, it is planned to develop an intelligent system based on artificial neural networks, which allows to control the motor functions of an anthropomorphic robot, including interacting with moving environmental objects. The developed system will be tested on the real anthropomorphic robot AR-600, which will be purchased in the framework of this project together with the control and training stand. The scientific novelty of the proposed project is to analyze and further use dynamic mechanisms, which determine the ability of living organisms to adaptively controlling motor systems in a complex and constantly changing environment, as a basis for constructing strategies and specific methods for adaptive control of biomorphic and anthropomorphic robots. The project should be brought to the stage of development of self-learning adaptive intelligent system of anthropomorphic robot, which will create a robotic platform that can operate in abnormal situations and provide locomotor function in a changing environment. The proposed project will carry out interrelated studies, including theoretical and numerical analysis of model neural networks, experimental radiophysical and neurophysiological studies, related to modeling and analysis (using machine learning and methods for processing large data) of activity of the brain and spinal cord, and human muscle activity (including copying using exoskeletons), analysis of interaction between the operator and the anthropomorphic robot basing on biofeedback principles, projecting and learning the systems of controlling by an anthropomorphic robot, basing on the methods and approaches of artificial intelligence.

Expected results
The following main results are expected after finishing the project: *There will be found the patterns, describing the adaptive dynamics of model networks, and formulated the proposals for the use of the results in the design of self-learning intelligent systems for controlling of anthropomorphic robotic systems. * A new model of the central rhythm generator, based on a heterogeneous ensemble of oscillators, will be proposed, characterized by the possibility of switching the generated rhythms corresponding to the coexisting stable dynamic modes of the ensemble. The resulting model will form the basis for the creation of an intelligent system that provides the locomotor activity of an anthropomorphic robot. * A set of experimental works will be carried out to record the human EEG data supplemented with data of muscle activity while controlling a full-sized anthropomorphic robot using a copying exoskeleton. Characteristic scenarios of brain activity and muscle activity will be revealed, ensuring the positioning of the human body, motor activity and interaction with dynamic objects. * A structured database of human neuronal and muscular activity, obtained during experiments on human control of the movements of an anthropomorphic robot, will be created for the training of intelligent control systems using machine learning technologies based on artificial neural networks. * Intelligent systems based on artificial neural networks and nonlinear dynamic models will be designed, allowing to automatically control the upper and lower extremities of the robot. To train the developed systems, the database of neural and muscular activity of a person obtained during experiments on human control of the movements of an anthropomorphic robot will be used. The expected results have significant scientific and social importance, caused by the modern world trends for the increasing introduction of robotic systems (including anthropomorphic ones) into the social sphere, science and production. In this context, scientific research within this project will be used the world's extensive scientific and technological backlog in this direction, as well as the team, offering this project, and, on its basis, developing new advanced concepts of adaptive intelligent control of anthropomorphic robotic systems in production, service applications, space technology, rescue operations and etc. As the result, there can be expected matching the announced results to the world level of research in the field of intelligent systems, analysis of large data and robotics. The scientific and technical products, resulting from the project and formed as systems for the intelligent control of robotic products, have great potential for further development and implementation in the development of robotic products at research and production enterprises in Russia, in particular JSC "Plant of Laboratory Equipment" Mechatronics and Robotics ", which is an industrial Partner of the project, SPA "Android Technics", LLC "Akrodim", engaged in the development and production of anthropomorphic robots and other enterprises in Russia. Therefore, it is likely to expect the formation of a vast scientific and technological reserve at the intersection of physics, robotics and intellectual technologies, based on the results of the project, which in the near future will be introduced into production and will make a significant contribution for the economic growth and social development of the Russian Federation in the field of robotics. It should be noted that the proposed project is unique in terms of experimental work, which is planned to be carried out using original robotic solutions. In this context, the possibility of conducting studies on real equipment is an important feature of the project, which determines the possibility of experimental verification and approbation of the developed solutions, which, in turn, significantly shortens the time required to introduce the results into the production process.


 

REPORTS


Annotation of the results obtained in 2020
A neural network based on the Hodgkin-Huxley biological neuron model was developed, capable of processing and classifying sensory information, while deciding on the type of input stimulus. The mechanism of operation of the proposed neural network is based on the formation of a chimera-like state when one part of the neurons is in an active state and generates spikes, while the other part is silent. The developed network consists of 100 Hodgkin-Huxley neurons, interconnected in accordance with a scale-free topology, and two output neurons. Initially, the main network is not connected to the output layer. In the process of training, these connections change over time, depending on the input signal, so that each of the output neurons is activated at a certain amplitude of the input signal, and the second remains at rest. Output neurons are interconnected by inhibitory connections. The developed network is able to determine the type of input pulse with close to 100% accuracy in a wide range of values of the amplitude of the external current pulse. Near the threshold value of the impulse, which corresponds to the average value of the amplitudes for which the training was carried out, the activation of both output neurons occurs equally, which corresponds to the ambiguity in determining the type of the presented stimulus and is in accordance with the known results of the experimental study of the perception of bistable images. An experimental radiophysical model of the central pattern generator (CPG) proposed at the previous stage of the project has been created. The constructed experimental prototype consists of an active van der Pol oscillator, which is connected to a chain of three linear dissipative oscillators. Maps of the oscillatory modes of the created CPG on the experimental planes of the control parameters have been constructed. The possibility of controlling the switching of the dynamic modes of the CPG, which are responsible for various types of locomotor activity, has been demonstrated. We have shown experimentally that when the frequency detuning of the self-sustained oscillator changes from the frequency of the three passive elements of the heterogeneous ensemble, hard switching occurs between the oscillatory modes. By shifting the oscillation phases of the ensemble elements, it is possible to control the gait of the robot by changing the values of the shift between the angles of rotation of the robot's limbs in the "hip", "knee" and "ankle" joints. The proposed CPG scheme has demonstrated its effectiveness in an experiment with a biped robot model for solving the problem of movement pattern generation that provides walking/running modes. We have shown experimentally that the developed CPG model demonstrates resistance to high noise level. It was found that switching of oscillatory modes in the investigated heterogeneous ensemble is observed even with a signal-to-noise ratio equal to -6 dB, which indicates the stability of the proposed CPG to noise and the prospects of its use in the tasks of controlling anthropomorphic robots. The characteristic features of the neural activity of the cerebral cortex and the trajectory of the gaze were revealed in the process of collecting unambiguous and ambiguous sensory visual information necessary for decision-making. We have shown that with ambiguity in sensory data, an increase in the energy of neural activity in theta and beta ranges is observed in the first 0.2 s after the presentation of a visual stimulus. The first point indicates that the subjects relied on internal representations (memory and contextual thinking), while an increase in energy in the beta range characterizes the activation of the mechanism for processing ambiguity and identifying relevant sensory data required for decision making. Analysis of the trajectory of eye movement using an Eye-Tracker showed that each orientation of the cube has its own characteristic pattern. The degree of ambiguity also affects the gaze trajectory. Thus, in the case of a highly ambiguous stimulus, the subject needs more sensory information to make a decision, which entails frequent switching of observation points. We have proposed an approach for the synthesis of neuro-fuzzy controllers using a modified genetic algorithm and an adaptive neuro-fuzzy inference system. The indicated approach is applied to the construction of Pareto-optimal controls for manipulator movements in an environment with static constraints. It is assumed that the manipulator also has limited mobility in the joints, which, in turn, imposes restrictions on possible movements. Suboptimal controllers were synthesized that minimize both the maximum possible deviations from the desired trajectory of movement and the maximum possible control moments created in the joints of the manipulator. The correctness of the developed controllers was tested on the model, and a comparison was made with the controllers built in the classical way using the semi-definite programming apparatus for the system obtained by linearizing the original along the desired trajectory of motion. The spatial-temporal structure of changes in the level of complexity of the signal of electrical activity of the human brain in the mu-rhythm range (8-14 Hz) was studied using recurrent analysis during the executions of the motor function. It was shown that the execution of hand movements is associated with a statistically significant increase in the regularity of the signal, localized in the region of the motor cortex with the dominant role of the contralateral hemisphere. A hypothesis was formulated that an increase in the regularity of the EEG signal can be interpreted from the standpoint of adaptation of the dynamics of local neuronal populations of the motor cortex to the performance of a motor task under the action of the mechanism of neural plasticity. As a consequence, we have shown that the regularity of fluctuations between assignments, i.e. at pre-stimulus temporal intervals, demonstrates statistically significant growth during the experiment. This may indicate healthy mechanisms for training the motor functions of the brain and adaptation of neural ensembles for more optimal activation during sequentially repeated movements. A method for assessing the complexity and long-term correlations in EEG signals associated with the execution of movements, based on the detrended fluctuation analysis of time series, is proposed. We have shown that movement of the non-dominant (left) hand determines relatively weak changes in the long-range correlations of EEG signals, as compared to the dominant (right) hand. A real-time control system for a two-legged walking robot has been developed without using a copying exoskeleton by analyzing the neural activity of the operator using the methods of recurrence quantitative analysis developed within the framework of the project. The classifier of EEG signals associated with the execution of arm and leg movements is based on a quantitative analysis of recurrent diagrams. The time dependence of the measure of determinism (DET), which has the property of contralaterality, was used as a criterion for classification. The main idea of the proposed classifier is that each type of movement is characterized by its own unique DET pattern. The fast speed of classification is ensured through the use of parallel computing technologies and the corresponding optimization of the algorithm. Signals of the neural activity of the operator's brain in the process of performing movements, first undergo preliminary processing, and then are fed to the input of the classifier, which determines the type of perfect movement (movement with the left/right arm/leg). Based on the information received, a control command is generated for the walking robot. We have shown the presence of a muscle pattern that occurs during training a person while maintaining balance. The muscle pattern includes four muscle pairs: the tibialis anterior muscle on the right leg and the triceps muscle of the calf on the right leg, the tibialis muscle on the right leg and the triceps muscle of the calf on the left leg, the tibialis muscle on the right leg and the quadriceps muscle of the thigh on the right leg, the triceps muscle of the calf on the right leg and the triceps muscle of the calf on the left leg. The results of statistical analysis of behavioral characteristics show a significant increase between sessions in the duration of the longest equilibrium segment and the total duration of all equilibrium segments. This conclusion is confirmed by statistical analysis of muscle activity, which demonstrates that a significant change in the correlation coefficients between 4 pairs of muscles of the detected pattern occurs only between experimental sessions 1 and 2. We have found that the number of muscles involved in the activation pattern is limited. It was shown that the pattern of muscle activation is asymmetric, which confirms the ability of the posture control system to use asymmetric patterns to maintain balance. A mathematical model was constructed for the first time, based on mechanical principles, taking into account the mechanisms of training a person while maintaining balance, muscle control in which is carried out on the basis of neural controllers with feedback, including the deviation of the platform angle, speed, acceleration, displacement of the center of mass of the body, afferent nerve delays and proprioceptive factor. The use of the developed model made it possible to identify a limited set of correlation patterns between the muscles of the legs of a person during training, in which the interaction between the muscles increases during the long-term performance of complex actions associated with maintaining balance. Neural activity in the sensorimotor area of the cerebral cortex shows significant changes in the process of maintaining balance, accompanied by a general decrease in activity in the O1, O2, Oz, Pz, Cpz EEG channels. In the course of multivariate analysis of variance, it was revealed that significant differences are achieved in the high frequency range of 23.75-40 Hz in the right and left parietal areas and in the central frontal area. It should be noted that this difference is achieved at the beginning of the session (200–240 seconds), while the energy in this cluster is maximum in the first session and decreases from session to session. On September 7-9, 2020, the IV International Scientific School "Dynamics of Complex Networks and their Application in Intellectual Robotics" (DCNAIR'2020) (https://events.innopolis.university/dcnair2020) was held at Innopolis University, in which 6 Russian and 9 foreign leading scientists (lecturers), as well as 75 participants - Russian and foreign young scientists under the age of 35 took part Selected DCNAIR 2020 papers was submitted in the IEEE Explore Library and indexed in Scopus.

 

Publications

1. Andreev A.V., Ivanchenko M.V., Pisarchik A.N., Hramov A.E. Stimulus classification using chimera-like states in a spiking neural network Chaos, Solitons and Fractals, V. 139, P. 110061 (year - 2020) https://doi.org/10.1016/j.chaos.2020.110061

2. Andreev A.V., Maksimenko V.A., Pisarchik A.N., Hramov A.E. Synchronization of interacted spiking neuronal networks with inhibitory coupling Chaos, Solitons and Fractals, - (year - 2020)

3. Golousov S., Savin S., Kurkin S., Badarin A., Khorev V., Hramov A., Klimchik A. Humanoid robot solving a task of balancing on a tilting platform Cybernetics and Physics, V. 9, I. 1, Pp. 5-12. (year - 2020) https://doi.org/10.35470/2226-4116-2019-8-4-282-286

4. Hramov A.E., Grubov V., Badarin A., Maksimenko V.A., Pisarchik A.N. Functional Near-Infrared Spectroscopy for the Classification of Motor-Related Brain Activity on the Sensor-Level Sensors, V. 20, I. 8, P. 2362 (year - 2020) https://doi.org/10.3390/s20082362

5. Khorev V.S., Badarin A.A., Grubov V.V., Maksimenko V.A., Kurkin S.A., Pisarchik A.N., Hramov A.E. Emerging muscle correlation pattern during training-improved performance of a postural balancing task Nonlinear Dynamics, - (year - 2020)

6. Kulminskiy D.D., Kurkin S.A., Ponomarenko V.I., Prokhorov M.D., Astakhov S.V., Hramov A.E. Central pattern generator based on self-sustained oscillator coupled to a chain of oscillatory circuits Nonlinear Dynamics, - (year - 2020)

7. Pavlov A.N., Pitsik E.N., Frolov N.S., Badarin A., Pavlova O.N., Hramov A.E. Age-Related Distinctions in EEG Signals during Execution of Motor Tasks Characterized in Terms of Long-Range Correlations Sensors, V. 20, No. 20, P. 5843 (year - 2020) https://doi.org/10.3390/s20205843

8. Pitsik E., Frolov N., Kraemer K.H., Grubov V., Maksimenko V., Kurths J., Hramov A. Motor execution reduces EEG signals complexity: Recurrence quantification analysis study Chaos: An Interdisciplinary Journal of Nonlinear Science, V. 30, I. 2, P. 023111 (year - 2020) https://doi.org/10.1063/1.5136246

9. Ponomarenko V.I., Kulminskiy D.D., Andreev A.V., Prokhorov M.D. Оценка амплитуды внешнего периодического воздействия при помощи малой спайковой нейронной сети в радиофизическом эксперименте Письма в Журнал технической физики, - (year - 2021)

10. Hramov A. E., Frolov N. S., Maksimenko V. A., Kurkin S. A., Kazantsev V. B., Pisarchik A. N. Функциональные сети головного мозга: от восстановления связей до динамической интеграции Успехи физических наук, - (year - 2020) https://doi.org/10.3367/UFNr.2020.06.038807

11. Andreev A., Pisarchik A. Classification of external signal by spiking neural network of bistable Hodgkin-Huxley neurons 4th Scientific School on Dynamics of Complex Networks and their Application in Intellectual Robotics, P. 31-33 (year - 2020) https://doi.org/10.1109/DCNAIR50402.2020.9216925

12. Andreev A.V., Frolov N.S., Alexandrova N.A., Chaban M.A. Control of dynamics of bistable neural network by an external pulse Proc. SPIE, V. 11459, P. 114590W (year - 2020) https://doi.org/10.1117/12.2563872

13. Andreev A.V., Malova N.A., Borovkova E.I., Frolov N.S. Interaction of bistable neurons leading to the complex network dynamics Proc. SPIE, V.11459, P. 114590V (year - 2020) https://doi.org/10.1117/12.2563866

14. Frolov N.S., Makarov V.V. Inference of functional dependence in coupled chaotic systems using feed-forward neural network Proc. SPIE, V. 11459, P. 114590X (year - 2020) https://doi.org/10.1117/12.2563980

15. Grishina D. S., Kupriyashkina N. M., Pavlova O. N., Runnova A. E., Pavlov A. N. Recognition of EEG patterns during mental intentions: a comparative study Proc. SPIE, V. 11459, P. 1145902 (year - 2020) https://doi.org/10.1117/12.2559689

16. Khorev V., Grubov V., Badarin A., Kurkin S. Dynamical analysis of the neural and equilibrium seeking movement activity 4th Scientific School on Dynamics of Complex Networks and their Application in Intellectual Robotics, Pp. 124-128 (year - 2020) https://doi.org/10.1109/DCNAIR50402.2020.9216815

17. Khorev V.S., Pushkarskaja D.D., Pitsik E.N., Kurkin S.A. The technique for determining on EMG signals the precursors of start of limb movement Proc. SPIE, V. 11459, P. 114590I (year - 2020) https://doi.org/10.1117/12.2563541

18. Kurkin S., Hramov A., Chholak P., Pisarchik A. Localizing oscillatory sources in a brain by MEG data during cognitive activity 2020 4th International Conference on Computational Intelligence and Networks (CINE), Pp. 1-4 (year - 2020) https://doi.org/10.1109/CINE48825.2020.234403

19. Maksimenko V., Khorev V., Grubov V., Badarin A., Hramov A.E. Neural activity during maintaining a body balance Proc. SPIE, V. 11459, P. 1145903 (year - 2020) https://doi.org/10.1117/12.2563533

20. - Способ классификации двигательной активности человека -, 2020127415 (year - )

21. - IV Международная научная школа DCNAIR 2020 Группа РНФ ВКонтакте, - (year - )

22. - IV Международная научная школа «Динамика сложных сетей и их применение в интеллектуальной робототехнике» (DCNAIR 2020) прошла в сентябре на базе Innopolis University в рамках проекта по гранту РНФ Группа РНФ в Facebook, - (year - )


Annotation of the results obtained in 2017
A model network of phase oscillators was developed, which topology is characterized by two scales of connectivity - the existence of a structure of coupled sub-networks. We have studied how the relationship between topologies of structural scales effects the dynamic regimes, in particular, on the implementation of such specific regimes as the chimera state. It is shown that a change in the connectivity of subnets allows to control the relation of the size of synchronous and asynchronous cluster on a global scale. The results obtained within the framework of this task will be used to develop a multiscale network of neural elements in the context of creating a central pattern generator. We numerically model one-layer and multilayer network consisting of Rulkov neurons coupled to each other by synaptical coupling. Couplings inside each layer are excitatory, but between layers a part of couplings are inhibitory and another one is excitatory. We discover the phenomenon of formation of space-time structures and investigate the influence of such parameters as network size, number of stimulated neurons, part of inhibitory couplings between layers and amplitudes of external stimulus and internal noise on it. We discover the phenomenon of coherent resonance depending from the parameters. A new mechanism of structural adaptation in the network of the Rulkov maps is developed on the basis of the spike-timing-dependent plasticity principle. It is shown that the developed model demonstrates a much more complex dynamics in comparison with the classical STPD mechanism. It was revealed that this difference is directly related to the type of structures formed in the process of adaptation: the implementation of an additional homeostatic term in the coupling equation led to the emergence of highly heterogeneous clusters in the structure and the appearance of the scale-free type of structure inherent in real neural ensembles of the brain. These results open the possibility of creating a central pattern generator, the characteristics of which adapt to the external conditions. Preliminary experimental studies are carried out with the registration of human neuronal activity during active movement to control an anthropomorphic hand prosthesis. 56 experimental measurements were performed with conditionally healthy volunteers. Various types of data analysis were carried out: spectral analysis with the help of wavelet spectra, cross-spectral analysis, estimation of phase coherence coefficients, which allowed to pre-select promising pairs of EEG signals for more detailed analysis, as well as frequency ranges of analysis. After choosing the parameters of the methods of analysis, a study of the directional coupling was carried out which made it possible to reveal the significant difference between symmetric EEG leads in opposite directions from the background values for both real and imaginary motions. This indicates the prospect of using the methods of the directional coupling analysis based on the simulation of phase dynamics for solving the problems of constructing the BCI. Using multifractal analysis, the structure of EEG signals recorded in untrained operators during motor activity was studied. On the basis of the obtained results, brain regions are identified for which reliable detection of differences between EEG signals acquired during hand movements and background brain electrical activity can be provided. It is shown that the highest efficiency of classification is achieved by using EEG recordings acquired in the frontal regions of the brain. The technique based on artificial neural networks for recognition and classification of EEG patterns corresponding to different types of movements has been developed, it has demonstrated high efficiency for untrained subjects: recognition accuracy up to 90-95%. Radial basis function network shows the best recognition results. Pre-filtering of the input EEG data using the low-pass filter significantly increases the recognition accuracy (on average by 10–20%), and the low-pass filter with a cutoff frequency 4 Hz shows the best results. We have demonstrated that when using signals from certain groups of electrodes consisting of 6-12 channels, the classification accuracy reaches a value close to the maximum. The last result is important from a practical point of view, because it shows the possibility of using more compact systems for EEG signals registration (with fewer electrodes) while maintaining the required recognition accuracy. First School for Young Scientists “Dynamics of Complex Networks and their Application in Intellectual Robotics” (DCNAIR 2017) was held on 20-22 November 2017 in Yuri Gagarin State Technical University of Saratov (http://www.sstu.ru/nauka/konferentsii/dinamika-slozhnykh-setey-i-ikh-primenenie-v-intellektualnoy-robototekhnike-dcnair-2017.html). DCNAIR 2017 has become an international platform for teaching and exchanging educational, scientific and technical ideas among specialists, especially young scientists and students, working in the field of studying complex networks and intelligent robotics. DCNAIR 2017 promoted active scientific cooperation, both at the Russian and international levels (http://www.sstu.ru/news/uchenye-predstavili-rezultaty-issledovaniy-na-mezhdunarodnoy-ploshchadke-po-intellektualnoy-robotote.html).

 

Publications

1. Alexander N. Pisarchik, Mariano Alberto García-Vellisca, Rider Jaimes-Reátegui, Francisco de Pozo-Guerrero Bistability in Hindmarsh-Rose neural oscillators induced by asymmetric electrical coupling CYBERNETICS AND PHYSICS, VOL. 6, NO. 3, P. 126–130 (year - 2017)

2. Andreev A.A., Makarov V.V., Runnova A.E., Pisarchik A.N., Hramov A.E Coherence resonance in stimulated neuronal network Chaos, Solitons and Fractals, 106, (2018) 80-85 (year - 2018) https://doi.org/10.1016/j.chaos.2017.11.017

3. Andreev A.V., Makarov V.V., Runnova A.E., Hramov A.E. Coherent resonance in neuron ensemble with electrical couplings CYBERNETICS AND PHYSICS, VOL. 6, NO. 3, P. 145–148 (year - 2017)

4. Frolov N.S., Koronovskii A.A., Makarov V.V., Maksimenko V.A., Goremyko M.V., Hramov A.E. Control of pattern formation in complex network by multiplexing CYBERNETICS AND PHYSICS, VOL. 6, NO. 3, P. 121–125 (year - 2017)

5. Grubov V.V., Musatov V.Yu., Maksimenko V.A., Pisarchik A.N., Hramov A.E. Development of intelligent system for classification of multiple human brain states corresponding to different real and imaginary movements CYBERNETICS AND PHYSICS, VOL. 6, NO. 3, P. 103–107 (year - 2017)

6. - Программа для ЭВМ для моделирования динамики многослойной адаптивной кооперативной сети с использованием технологий параллельных вычислений -, - (year - )

7. - Программа для ЭВМ для моделирования динамики многослойной адаптивной конкурентной сети с использованием технологий параллельных вычислений -, - (year - )

8. - Программа для ЭВМ для расчёта вейвлет-преобразования ЭЭГ сигнала с использованием параллельных вычислений на базе CUDA NVideo -, 2017618971 (year - )

9. - В Саратовском техническом университете взялись очеловечить робота Телеканал Россия 1, Программа Вести в 11:00 от 12.09.17 (year - )

10. - Саратовские ученые научат роботов сомневаться и принимать решения Сайт Российского научного фонда, 30 августа 2017 г. (year - )

11. - Саратовские ученые получили 120 млн руб. на создание искусственного интеллекта роботов Федеральное государственное унитарное предприятие «Информационное телеграфное агентство России (ИТАР-ТАСС)», 12 июля 2017 г. (year - )

12. - В лабораториях Саратовского техуниверситета установлено новое оборудование Сайт 4science, 5 августа 2017 г. (year - )


Annotation of the results obtained in 2018
For the first time, a system consisting of an active oscillator that demonstrates quasi-harmonic self-oscillations (a model of the Van der Pol oscillator) and two passive linear dissipative oscillators coupled with it is proposed as a model of a central rhythm generator (CRG). The signals taken from the generator and the contour are used as signals controlling the movements of the legs in the hip and knee joints of the robot. The CRG regime determines the speed of movement of the robot, gait and the regime of "running" or "walking." A mathematical model of a CRG was developed, and with its use a study of the dynamics of the CRG was performed. It was found that the movement along a part of the plane of the parameters of an external action, corresponding to synchronization through suppression, allows one to realize a soft transition from one oscillation regime to another, which corresponds to the running/walking switch. A numerical and experimental study of the dynamics of a network consisting of delay-coupled neural-like FitzHugh-Nagumo oscillators was carried out. A radiophysical setup was created for experimental research of possibility of controlling oscillatory regimes in an ensemble of neural-like oscillators coupled by tuned connections. It is shown that the use of an adaptively controlled delayed coupling allows one to solve the problem of in-phase synchronization of nonidentical neural-like oscillators in a wide range of control parameters. For an experimental study of a soft-excited self-sustained oscillator modeled by the Van der Pol equation, a radiophysical setup was developed and constructed, which includes a generator with cubic nonlinearity and two additional oscillatory circuits connected to the generator circuit via capacitive coupling. The dependence of the amplitude of oscillations in the main circuit on the capacitance of the capacitor in the main generator, which is responsible for the detuning of the frequencies of oscillator and additional circuits, is experimentally investigated. It is shown that by varying the frequency of self-sustained oscillations of the main generator, one can control the multistable oscillatory regimes in the system and observe hard transitions between them. In the course of this work, the effects of synchronization and the formation of spatio-temporal structures in multilayer networks of phase and neuron-like oscillators were numerically studied. A new phenomenon - the macroscopic chimera-like state of the network - was discovered and studied in detail. This effect consists in dividing the initially identical layers of the network into subgroups that demonstrate different dynamic modes under their interaction on the macro-level. Such states may be associated with the state of doubt of the brain neural network during the making of ambiguous decisions. Analysis of the neural activity of the human brain was conducted during the observation of bistable images and the decision on their interpretation. The magnetoencephalographic experiment data was processed using an artificial neural network trained to classify the states of perception of low ambiguous visual stimuli. Objective criteria were introduced to describe the process of information processing and decision making in a situation with ambiguity of choice. Using these criteria, an algorithm is proposed for an intelligent decision-making system based on the processing of external sensory information. According to the results of the experimental work, according to the developed design, an array of homogeneous experimental data was formed, containing sets of multichannel electroencephalography and myographies recorded from 20 conditionally healthy subjects. A unified algorithm for processing complex arrays of electroencephalographic (EEG) and electromyographic (EMG) signals has been developed. The processing complex is based on the allocation of a person’s motor activity and its objective classification according to myographic signals. It is shown that the use of methods of energy and skeleton estimations of continuous wavelet transform allows to establish an exact connection between the occurrence of muscle activity (increase in the amplitude of EMG) and EEG signals in the sensorimotor recording area. The use of the method of multiscale analysis based on discrete wavelet transform allows to reliably distinguish the signals of the electrical activity of the brain when performing movements. To highlight the general patterns inherent in the vertical posture of the subjects, energy assessments were made of the main neurophysiological modes of the oscillatory activity of the human brain in the moments of the implementation of various types of movements. To minimize the disturbance components caused by involuntary tension of the cervical and shoulder muscle groups, a special algorithm has been proposed, which includes separate recording of signals from several electrodes located on the back of the muscles, and the subsequent activation of filtering based on the calculation of empirical Hilbert-Huang transform modes at times of enhancement the amplitude level of high-frequency activity in the recorded myographic signals, leading to a specific distortion of brain activity. It was revealed that the transition to the vertical position of the subject, close to the natural, causes a significant complication of the frequency structure of the dynamics of the electrical activity of the brain, making the generation of high-frequency oscillations characteristic of the sensorimotor cortex in the classical neurophysiological samples less pronounced. We have proposed methods for control an anthropomorphic prosthetic arm using neural activity signals and muscle activity via direct decoding of biological signals without a copying exoskeleton. It has been shown that movement of the arm causes specific pattern formation on the signal of the corresponding muscle electrical activity, the shape of which keeps similar from one move to other. This allows us to automatically detect the exact moment when the movement starts using preliminary specified threshold. We have shown, that the optimal threshold value allows to correctly detect 100% of movements with 10% false detection rate. It has been shown that the characteristic motor-related patterns of brain electrical activity can be detected with an accuracy of 92% and with 5% false detection rate. When movements are performed under conditions of additional external stimulation, the frontal brain areas exhibit activity increase. It is associated with an attention on the external stimulus. Less than a second before the beginning of the movement and during its execution, both in the case of voluntary motor activity and in the case of additional external stimuli, there is a local decrease in the activity of 8-12 Hz in the left part of the central-frontal cortex (for movement with the right hand). At the same time, in the case of the presence of an additional external stimulus, there is a persistence of activity in the frontal areas and suppression of the activity of 8-12 Hz in the occipital cortex, which also indicates the concentration of attention during sensory information processing. We have proposed experimental design for analysis of electrical signals of brain (EEG) and muscle activity (EMG) during anthropomorphic robot control using copying exoskeleton. Spatial and frequency properties of neuronal activity patterns have been revealed using artificial neural network (ANN) and time-frequency methods during motor commands generation for controlling movements of robot legs. It has been shown that the most accurate classification of motor-related patterns can be reached by using EEG signals in frontal and parietal lobes. Moreover, the most informative frequency range is shown to be 1-12 Hz. Unique and common features of EEG signals, which are associated with control of arms and legs movements, have been analyzed using ANN. It has been shown, that ANN effectively (average accuracy 95%) detects the differences between these patterns when high-frequency EEG component is considered (>30 Hz). In the case when the low-frequency components (<4 Hz) are taken into account, the average classification accuracy decreases to 70%. The intellectual system for anthropomorphic robot control has been developed on a basis of correlated processes of brain and muscle activity. In order to detect EEG motor patterns and find correlation between them and corresponding patterns of EMG, the recurrent ANN has been used. The average recognition accuracy of motor commands is shown to be 90%. Second School for Young Scientists “Dynamics of Complex Networks and their Application in Intellectual Robotics” (DCNAIR 2018) was held on 8-10 October 2018 in Yuri Gagarin State Technical University of Saratov (http://www.sstu.ru/nauka/konferentsii/dcnair-2018-ru.html). DCNAIR 2018 has become an international platform for teaching and exchanging educational, scientific and technical ideas among specialists, especially young scientists and students, working in the field of studying complex networks and intelligent robotics. DCNAIR 2018 promoted active scientific cooperation, both at the Russian and international levels (http://www.sstu.ru/news/v-sgtu-proshla-shkola-konferentsiya-po-intellektualnoy-robototekhnike.html).

 

Publications

1. Andreev A.V., Runnova A.E., Pisarchik A.N. Numerical simulation of coherent resonance in a model network of Rulkov neurons Proc. SPIE, V.10717, No. 107172E, P. 1-6 (year - 2018) https://doi.org/10.1117/12.2315092

2. Frolov N.S., Maksimenko V.A., Makarov V.V., Kirsanov D.V., Hramov A.E., Kurths J. Macroscopic chimeralike behavior in a multiplex network Physical Review E, Vol. 98, Iss. 2, No. 022320 (year - 2018) https://doi.org/10.1103/PhysRevE.98.022320

3. Frolov N.S., Pisarchik A.N. Diagnostics of the Brain Neural-Ensemble States Using MEG Records and Artificial Neural-Network Concepts Technical Physics Letters, Vol. 44, No. 5, pp. 441–444 (year - 2018) https://doi.org/10.1134/S1063785018050176

4. Hramov A.E, Selskii A.O., Egorov I.V. Nonlinear correlation method for the separation of couplings in EEG experiments with neural ensembles Proc. SPIE, Vol. 10493, No. 104931C, P. 1-6 (year - 2018) https://doi.org/10.1117/12.2291669

5. Hramov A.E., Frolov N.S., Maksimenko V.A., Makarov V.V., Koronovskii A.A., Garcia-Prieto J., Anton-Toro L.F., Maestu F., Pisarchik A.N. Artificial neural network detects human uncertainty Chaos, V. 28, Issue 3, No. 033607 (year - 2018) https://doi.org/10.1063/1.5002892

6. Kurkin S., Pitsik E., Musatov V., Runnova A., Hramov A. Artificial Neural Networks as a Tool for Recognition of Movements by Electroencephalograms In Proceedings of the 15th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2018), Vol. 1, P. 166-171 (year - 2018) https://doi.org/10.5220/0006860201660171

7. Makarov V.V. ,Zhuravlev M.O., Runnova A.E., Protasov P., Maksimenko V.A., Frolov N.S., Pisarchik A.N., Hramov A.E. Betweenness centrality in multiplex brain network during mental task evaluation Physical Review E, - (year - 2018)

8. Makarov V.V., Kirsanov D., Goremyko M., Andreev A., Hramov A.E. Nonlinear dynamics of the complex multi-scale network Proc. SPIE, V. 10717, No. 1071729, P. 1-7 (year - 2018) https://doi.org/10.1117/12.2315095

9. Makarov V.V., Kirsanov D.V., Frolov N.S., Maksimenko V.A., Li X., Wang Z., Hramov A.E., Boccaletti S. Assortative mixing in spatially extended networks Scientific Reports, V. 8, No. 13825, P. 1-8 (year - 2018) https://doi.org/10.1038/s41598-018-32160-4

10. Makarov V.V., Kundu S., Kirsanov D.V., Frolov N.S., Maksimenko V.A., Ghosh D., Dana S.K., Hramov A.E. Multiscale interaction promotes chimera states in complex networks Communications in Nonlinear Science and Numerical Simulation, Vol. 71, P. 118–129 (year - 2019) https://doi.org/10.1016/j.cnsns.2018.11.015

11. Maksimenko V., Runnova A., Pchelintseva S., Efremova T., Zhuravlev M., Pisarchik A. Analysis of the features of untrained human movements based on the multichannel EEG for controlling anthropomorphic robotic arm Proc. SPIE, Vol. 10717, No. 107171L, P. 1-7 (year - 2018) https://doi.org/10.1117/12.2314686

12. Maksimenko V.A., Hramov A.E., Grubov V.V., Nedaivozov V.O., Makarov V.V., Pisarchik A.N. Nonlinear effect of biological feedback on brain attentional state Nonlinear Dynamics, P. 1-17 (year - 2018) https://doi.org/10.1007/s11071-018-4668-1

13. Maksimenko V.A., Kurkin S.A., Pitsik E.N., Musatov V.Yu., Runnova A.E., Efremova T.Yu., Hramov A.E., Pisarchik A.N. Artificial Neural Network Classification of Motor-Related EEG: An Increase in Classification Accuracy by Reducing Signal Complexity Complexity, V. 2018, No. 9385947, P. 1-10 (year - 2018) https://doi.org/10.1155/2018/9385947

14. Maksimenko V.A., Pavlov A., Runnova A.E., Nedaivozov V., Grubov V., Koronovskii A., Pchelintseva S.V., Pitsik E., Pisarchik A.N., Hramov A.E. Nonlinear analysis of brain activity, associated with motor action and motor imaginary in untrained subjects Nonlinear Dynamics, V. 91, Issue 4, P. 2803–2817 (year - 2018) https://doi.org/10.1007/s11071-018-4047-y

15. Maksimenko V.A., Runnova A.E., Frolov N.S., Makarov V.V., Nedaivozov V., Koronovskii A.A., Pisarchik A., Hramov A.E. Multiscale neural connectivity during human sensory processing in the brain Physical Review E, V. 97, Issue 5, No. 052405 (year - 2018) https://doi.org/10.1103/PhysRevE.97.052405

16. Pavlov A.N., Runnova A.E., Maksimenko V.A., Grishina D.S., Hramov A.E. Detection of EEG-patterns associated with real and imaginary movements using detrended fluctuation analysis Proc. SPIE, V.10493, No. 1049315, P. 1-5 (year - 2018) https://doi.org/10.1117/12.2291878

17. Pavlov A.N., Runnova A.E., Maksimenko V.A., Pavlova O.N., Grishina D.S., Hramov A.E. Распознавание движений руки по сигналам электроэнцефалограммы на основе флуктуационного анализа Письма в ЖТФ, 2019, том 45, вып. 4 (year - 2019) https://doi.org/10.21883/0000000000

18. Pavlov A.N., Runnova A.E., Maksimenko V.A., Pavlova O.N., Grishina D.S., Hramov A.E. Detrended fluctuation analysis of EEG patterns associated with real and imaginary arm movements Physica A, V. 509, P. 777-782 (year - 2018) https://doi.org/10.1016/j.physa.2018.06.096

19. Pisarchik A.N., Huerta-Cuellar G., Kulp C.W. Statistical analysis of symbolic dynamics in weakly coupled chaotic oscillators Communications in Nonlinear Science and Numerical Simulation, V. 62, P. 134-145 (year - 2018) https://doi.org/10.1016/j.cnsns.2018.02.025

20. Pitsik E.N., Makarov V.V., Kirsanov D.V., Frolov N.S., Goremyko M., Li X., Wang Z., Hramov A.E., Boccaletti S. Inter-layer competition in adaptive multiplex network New Journal of Physics, V. 20, No. 075004, P. 1-9 (year - 2018) https://doi.org/10.1088/1367-2630/aad00d

21. Pitsik E.N., Makarov V.V., Nedaivozov V.O., Kirsanov D.V., Goremyko M.V. Self-organization in multilayer network with adaptation mechanisms based on competition Proc. SPIE, V.10717, No.107172B, P. 1-6 (year - 2018) https://doi.org/10.1117/12.2315120

22. Ponomarenko V.I., Kul’minskii D.D., Prokhorov M.D. An Experimental Study of Synchronization of Nonidentical Neuronlike Oscillators with an Adaptive Delayed Coupling Technical Physics Letters, Vol. 44, Iss. 9, P. 761-764 (year - 2018) https://doi.org/10.1134/S1063785018090109

23. - Программа ЭВМ обеспечения сложного дизайна нейрофизиологического эксперимента в области исследования двигательной активности различных типов при синхронной электроэнцефалографической регистрации у человека -, 2018612999 (year - )

24. - Способ идентификации состояния сомнений человека по данным активности головного мозга -, - (year - )

25. - Способ классификации сигналов ЭЭГ при воображении двигательной активности у нетренированного оператора -, - (year - )

26. - Программа для моделирования динамики многомасштабной сети -, 2018613337 (year - )

27. - Интерфейс для распознавания и визуализации воображаемых движений рукой -, 2018612855 (year - )

28. - Программный модуль для поиска и классификации состояний головного мозга в процессе принятия решения -, 2018662581 (year - )

29. - РАЗМИНКА ДЛЯ УМА. ОТ "ИГРЫ В БИСЕР" - К РЕАЛЬНЫМ ПРОБЛЕМАМ РОБОТОТЕХНИКИ И НЕЙРОТЕХНОЛОГИЙ газета "Поиск", № 16(2018) (year - )

30. - Российские биологи выяснили, как мозг управляет фантомными конечностями РИА НАУКА, - (year - )

31. - С помощью электроэнцефалограммы сравнили реальное и воображаемое движение человека Индикатор, - (year - )

32. - Ученые смогли отличить реальные и воображаемые движения человека Газета.ру, - (year - )


Annotation of the results obtained in 2019
A neural network of 100 Hodgkin-Huxley neurons has been proposed and implemented. The network demonstrates behavior similar to the behavior of the human brain during decision making in conditions characterized by a lack or ambiguity of information. The operative mechanism of the proposed neural network is based on a chimeric-like state formation, when one part of the neurons is in the active state and generates spikes, while the second part is inactive. The possibility of controlling this state using an external current pulse is shown. This feature allows classifying various states. In the process of analyzing the operation of the network, it is shown that in conditions with a low degree of ambiguity, the neural network unambiguously makes decisions. With increasing ambiguity, it begins to “doubt”, but is more likely to make a decision in the direction of the orientation of the cube with a greater intensity of faces. With maximum ambiguity, competition between output neurons with equal probability can lead to the activation of each of them at the end of an ambiguous stimulus presentation. The role of the internal noise influence in the coding network of sensory information on the decision-making process was discovered: on the one hand, the presence of noise extends the boundaries of the ambiguity domain in decision-making, but on the other hand, it speeds up the decision-making process in the case of uniquely interpreted sensory information. Thus, the effects of stochastic and coherent resonance when adding noise to the original sensor signal can have a positive effect on the decision-making process in an artificial intelligent system. Based on the results of experimental studies, it has been suggested that the search for a solution to the problem of classifying ambiguous information can lead to an increase in the dimension of the feature space. From the point of view of an intelligent system, this requires a multilayer convolution of the input information signal to extract several levels of features. A mathematical model based on the interaction between two networks of neuron-like Hodgkin-Huxley elements was constructed. The model describes the process of perception and processing of sensory information and decision making via analysis of experimental data on brain electrical activity using network theory methods. One network preprocesses and encodes incoming sensory information into a sequence of neural spikes, the other network is involved in processing this signal. It is shown that a change in image contrast corresponds with a change in the degree of correlation of the recorded signals of brain activity is observed. The highest correlation is achieved in the image contrast range of 0.4-0.7. An analysis of functional connections of the experimental data on the brain electrical activity was performed that for this range of contrasts. This analysis demonstrates the characteristic formation of a distributed network structure of functional connections involving neural populations of the prefrontal and parietal regions of the cerebral cortex. A similar local increase in the correlation of neuron activity is achieved due to the influence of internal noise and is possible only at certain values of its intensity (coherent resonance effect). An original approach is proposed for the experimental study of an ensemble of neural-like oscillators, in which the radio-engineering installation provides the possibility of setting both constant and adaptive interaction between generators. This connection can be of any kind, and the change of connections inside the ensemble occurs in real time. Using an example of a system of two mutually connected non-identical oscillators and a circle consisting of 10 identical Fitzhugh-Nagumo oscillators, it is shown that the introduction of adaptive delayed coupling can provide a phase synchronization of the oscillators. This phase synchronization occurs even in the case of a large detuning of their parameters and in cases where a constant coupling cannot provide phase synchronization of all non-identical elements of the ensemble even with large values of the coupling coefficients. The experimental results are in good agreement with the results of numerical studies. It is shown that the adaptive connection of oscillators is promising in the development of CPG responsible for motion control. To implement various types of robot gait and switch between them, it is necessary to provide different types of synchronization between the ensemble oscillators, differing in the magnitude of the phase shift of their oscillations. A model of the central pattern generator (CPG) is proposed in the form of a self-excited oscillator with soft excitation, which is modeled by the Van der Pol equation and has two additional oscillatory circuits connected with the generator circuit via capacitive coupling. It is shown that this model is promising for the generation of gait of an anthropomorphic robot compatible of switching the gait mode. As a result of a detailed bifurcation analysis, it was found that applying an external harmonic effect to the developed system of the CPG in the coexistence mode of two oscillatory modes with different phase ratios allows switching between limiting cycles existing in the phase space of the system. The discovered structure of the space of external influence parameters allows switching between oscillatory modes in a multistable system by applying an external harmonic force to it and controlling its parameters. The created model was reproduced in the radiophysical experiment with the structure of the plane of the parameters corresponding with the mathematical model of the CPG. During the experiment, it was also established that the vibrational mode switching is observed in the system even with a signal-to-noise ratio of 0 dB. A CPG model is proposed that implements switching between the four types of locomotion. The model is based on an ensemble of heterogeneous oscillators. Van der Pol oscillator was chosen as an active oscillator, and the other elements of the ensemble were three linear dissipative oscillators consisting of inductor and capacitor. The elements of the ensemble were connected in a chain through capacitors of the same denomination. The phase ratio between the oscillations of the active and three passive oscillators corresponds to the ratio between the angles of rotation of the robot limbs in the "hip", "knee" and "ankle" joints. Changing the magnitude of the oscillators frequency detuning between two elements of the heterogeneous ensemble leads to a hard switch between the four oscillatory modes. The observed modes are stable and resistant to variations of other parameters, and can be used to generate various types of locomotion. A new model of walking primitives generator is also proposed. The model is based on the use of virtual holonomic connections using a motion generator that does not include an explicit function of the generalized coordinates. A method of stabilizing motion is shown. The method is based on reducing the system to a zero submanifold due to feedback on the residual of virtual connections and their speeds. A database was created consisting of neural and muscle activity signals of a person, corresponding to the persons execution of motor commands (both real and imaginary). The database contains the original EMG, EEG signals, as well as the time-frequency characteristics calculated for them. EEG and EMG signals were filtered using a 5th order Butterworth bandpass filter in the ranges 1-100 Hz and 10-100 Hz. The EEG trials were filtered in the ranges of mu- and beta-bands, which are the most informative in the analysis of the electrical activity of the brain, corresponding to real and imaginary movements. In addition, the database contains the time-frequency characteristics of each trial, such as the wavelet surfaces of EEG trials and the averaged wavelet energies of mu and beta rhythms. A control system for a two-legged walking robot without the use of a copying exoskeleton was developed using an analysis of the human neural activity. The methods were developed for the quantitative analysis of recurrence diagrams. Signals were recorded with an electroencephalograph from the operator in the process of performing movements. The signals were pre-processed to reduce noise and remove artifacts. Then signals were given to the classifier, which determines the type of perfect movement (movement with the left / right hand / foot). Based on the information received, a control command was generated for the walking robot. As a criterion for classification, we used the time dependence of the determinism measure (determinism, DET), which has a pronounced contralaterality property. The interaction between the blocks of the server part of the robot control interface is implemented using the execution queue, in which the execution commands are placed. An intelligent system was created, trained using EMG and EEG signals. The system automatically maintains the balance of a walking robot when the surface tilt varies over time. At the first stage, experiments were conducted to record the neural and muscle activity of a person while maintaining equilibrium on a balance platform. It is shown that during maintaining equilibrium on the balance platform, there is a change in the distribution of the values of the angle of the balance platform from the equilibrium position in each of the three experimental sessions. In the last session, the distribution maximum shifts to the region of small deviations, which indicates an increase in the time the platform is in equilibrium. This indicates a person’s adaptation to changing conditions. Studies have been conducted with an anthropomorphic walking robot. It should be noted that for particular parameters of the experiment, the behavior of the robot in a mathematical modeling environment is close enough to human behavior in full-scale experiments with a balance platform. It was shown that the implementation of feedback on the position of the center of mass is not enough to obtain the peculiarities of the robot’s movement close to that observed in experiments with subjects, while the introduction of an additional feedback channel on the balance-platform orientation made it possible to obtain results that are qualitatively similar to experimental data. On September 9-11, 2019, the III International School of Young Scientists “Dynamics of Complex Networks and Their Application in Intellectual Robotics” (DCNAIR 2019) was held at Innopolis University, in which 5 Russian students took part and 8 foreign academic lecturers, as well as 82 students - Russian young scientists under the age of 35 inclusive (https://lomonosov-msu.ru/rus/event/5553/). DCNAIR 2019 selected content is hosted on the IEEE Explore and indexed on Scopus and Web of Science.

 

Publications

1. Andreev A.V., Frolov N.S., Pisarchik A.N., Hramov A.E. Chimera state in complex networks of bistable Hodgkin-Huxley neurons Physical Review E, V. 100, No. 022224 (year - 2019) https://doi.org/10.1103/PhysRevE.100.022224

2. Andreev A.V., Pitsik E.N., Makarov V.V., Pisarchik A.N., Hramov A.E. Dynamics of map-based neuronal network with modified spike-timing-dependent plasticity The European Physical Journal Special Topics, V. 227, I. 10-11, P. 1029–1038 (year - 2018) https://doi.org/10.1140/epjst/e2018-800036-5

3. Chholak P., Niso G., Maksimenko V.A., Kurkin S.A., Frolov N.S., Pitsik E.N., Hramov E.A., Pisarchik A.N. Visual and kinesthetic modes affect motor imagery classification in untrained subjects Scientific Reports, V. 9, No. 9838 (year - 2019) https://doi.org/10.1038/s41598-019-46310-9

4. Frolov N., Maksimenko V., Lüttjohann A., Koronovskii A., Hramov A. Feed-forward artificial neural network provides data-driven inference of functional connectivity Chaos, V. 29, N. 091101 (year - 2019) https://doi.org/10.1063/1.5117263

5. Frolov N.S., Hramov A.E. From theory to experimental evidence: Comment on “Chimera states in neuronal networks: A review” by S. Majhi, B.K. Bera, D. Ghosh and M. Perc. Physics of Life Reviews, V. 28, P. 125-127 (year - 2019) https://doi.org/10.1016/j.plrev.2019.02.006

6. Frolov N.S., Maksimenko V.A., Khramova M.V., Pisarchik A.N., Hramov A.E. Dynamics of functional connectivity in multilayer cortical brain network during sensory information processing The European Physical Journal Special Topics, V. 228, I. 11, P. 2381–2389 (year - 2019) https://doi.org/10.1140/epjst/e2019-900077-7

7. Grishina D.S., Pavlov A.N., Pavlova O.N., Runnova A.E. Use of Wavelets for Recognizing Types of Motion by Means of Data on the Electrical Activity of the Brain Technical Physics Letters, V. 45, I. 8, P. 820-822 (year - 2019) https://doi.org/10.1134/S1063785019080224

8. Hramov A.E., Maksimenko V.A., Koronovskii A., Runnova A.E., Zhuravlev M., Pisarchik A.N., Kurths J. Percept-related EEG classification using machine learning approach and features of functional brain connectivity Chaos, V. 29, N. 093110 (year - 2019) https://doi.org/10.1063/1.5113844

9. Khorev V., Maksimenko V., Kurkin S.A., Badarin A., Antipov V. EEG activity during balance platform test in humans Cybernetics and Physics, V. 8, N. 3. P. 132-136 (year - 2019)

10. Khorev V.S., Maksimenko V.A., Pitsik E.N., Runnova A.E., Kurkin S.A., Hramov A.E. Анализ двигательной активности с использованием сигналов электромиограмм Информационно-управляющие системы, № 3б с. 114-120 (year - 2019) https://doi.org/10.31799/1684-8853-2019-3-114-120

11. Kulminskiy D.D., Ponomarenko V.I., Prokhorov M.D., Hramov A.E. Synchronization in ensembles of delay-coupled nonidentical neuronlike oscillators Nonlinear Dynamics, V. 98, N. 1, P. 735-748 (year - 2019) https://doi.org/10.1007/s11071-019-05224-x

12. Pavlov A.N., Grishina D.S., Runnova A.E., Maksimenko V.A., Pavlova O.N., Shchukovsky N.V., Hramov A.E., Kurths J. Recognition of electroencephalographic patterns related to human movements or mental intentions with multiresolution analysis Chaos, Solitons & Fractals, V. 126, P. 230-235 (year - 2019) https://doi.org/10.1016/j.chaos.2019.06.016

13. Pisarchik A. N., Chholak P., Hramov A. E. Brain noise estimation from MEG response to flickering visual stimulation Chaos, Solitons & Fractals: X, V. 1, N. 100005 (year - 2019) https://doi.org/10.1016/j.csfx.2019.100005

14. Pisarchik A.N., Maksimenko V.A., Andreev A.V, Frolov N.S., Makarov V.V., Zhuravlev M.O., Runnova A.E., Hramov A.E. Coherent resonance in the distributed cortical network during sensory information processing Scientific reports, V. 9, N. 18325 (year - 2019) https://doi.org/10.1038/s41598-019-54577-1

15. Andreev A.V., Pisarchik A.N. Mathematical simulation of coherent resonance phenomenon in a network of Hodgkin-Huxley biological neurons Proceedings SPIE, V. 11067, N. 1106708 (year - 2019) https://doi.org/10.1117/12.2527709

16. Frolov N., Zhukov V., Pisarchik A. Multistability of macroscopic states in multilayer multiplex network 2018 2nd School on Dynamics of Complex Networks and their Application in Intellectual Robotics (DCNAIR), Saratov, P. 35-37 (year - 2018) https://doi.org/10.1109/DCNAIR.2018.8589199

17. Frolov N.S., Hramov A.E. Multilayer perceptron reveals functional connectivity structrure in thalamo-cortical brain network 2019 3rd School on Dynamics of Complex Networks and their Application in Intellectual Robotics (DCNAIR), p. 53-54 (year - 2019) https://doi.org/10.1109/DCNAIR.2019.8875539

18. Hramov A., Pitsik E., Chholak P., Maksimenko V., Frolov N., Kurkin S., Pisarchik A. A MEG Study of Different Motor Imagery Modes in Untrained Subjects for BCI Applications In Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics, V. 1, P. 188-195 (year - 2019) https://doi.org/10.5220/0007810001880195

19. Hramov A.E., Pisarchik A.N. Kinesthetic and visual modes of imaginary movement: MEG studies for BCI development 2019 3rd School on Dynamics of Complex Networks and their Application in Intellectual Robotics, DCNAIR 2019, N. 8875521, P. 66-68 (year - 2019) https://doi.org/10.1109/DCNAIR.2019.8875521

20. Khorev V., Borovkova E., Runnova A., Zhuravlev M., Efremova T., Karavaev A. Detection of Couplings Between the EEG Signals During the Limb Movements 2018 2nd School on Dynamics of Complex Networks and their Application in Intellectual Robotics (DCNAIR), Saratov, p. 66-68 (year - 2018) https://doi.org/10.1109/DCNAIR.2018.8589057

21. Kirsanov D.V. Emergence of macroscopic chimera states in multilayer multiplex network Proceedings SPIE, V. 11067, N. 110670R (year - 2019) https://doi.org/10.1117/12.2527692

22. Kurkin S., Khorev V., Pitsik E., Maksimenko V., Hramov A. The Approach to the Detection of the Movement Precursor by Electromyographic Signals In Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics, V. 1, P. 276-280 (year - 2019) https://doi.org/10.5220/0007916502760280

23. Kurkin S.A., Pitsik E.N., Frolov N.S. Artificial intelligence systems for classifying EEG responses to imaginary and real movements of operators Proceedings SPIE, V. 11067, N. 1106709 (year - 2019) https://doi.org/10.1117/12.2527730

24. Pitsik E. Recurrence plot structure of motor-related human EEG 2019 3rd School on Dynamics of Complex Networks and their Application in Intellectual Robotics (DCNAIR), P. 139-141 (year - 2019) https://doi.org/10.1109/DCNAIR.2019.8875533

25. Pitsik E., Makarov V., Boccaletti S. Pattern formation in spatially distributed networks via spatially correlated preferential attachment Proceedings SPIE, V. 11067, N. 110670U (year - 2019) https://doi.org/10.1117/12.2527748

26. - Устройство для определения в режиме реального времени степени концентрации внимания оператора при восприятии и обработке информации -, Патент на изобретение № 2704562 (year - )

27. - Способ идентификации состояния сомнений человека по данным активности головного мозга -, Патент на изобретение № 2688320 (year - )

28. - Способ классификации сигналов ЭЭГ при воображении двигательной активности у нетренированного оператора -, Патент на изобретение № 2682492 (year - )

29. - Выделение функциональных связей данных головного мозга с использованием вейвлетного преобразования и рекуррентного анализа временных рядов -, № 2019661069 (year - )

30. - Программа по распознаванию начала движения по миографическим данным -, № 2019661004 (year - )

31. - Программа для ЭВМ моделирования химерного состояния в сети связанных нейронов Ходжкина-Хаксли -, № 2019660993 (year - )

32. - Программа для ЭВМ моделирования классификатора на основе формирования химерного состояния в сети связанных нейронов Ходжкина-Хаксли -, № 2019660992 (year - )

33. - Программа для вычисления мер количественного анализа рекуррентных диаграм вейвлетной энергии мю-ритма многоканальных ЭЭГ-сигналов, соответствующих реальным движениям руки -, № 2019661002 (year - )

34. - Люди по-разному представляют, как движется их тело, подтвердили ученые Газета "Поиск", - (year - )

35. - Люди представляют движения собственного тела по-разному Indicator, - (year - )

36. - Люди по-разному представляют, как движется их тело Газета Коммерсантъ, - (year - )

37. - Ученые: люди по-разному представляют, как движется их тело Газета.ru, - (year - )