Hi

Just Test Blog!

Particle Swarm Optimization (PSO)

My PRESENTATION

Abstract: This paper presents main paradigms of computational intelligence (CI) techniques and emphasizes their importance in understanding complex systems and designing proactive adaptive systems in uncertain, unknown, and dynamic environment. Several novel applications of hybrid CI techniques proposed by the author in engineering, manufacturing, biomedical and health care systems as well as engineering education are discussed. The experiences of presenting CI as a course and summer projects are also presented. The importance of introducing the CI techniques and their multidisciplinary applications as a senior level interdisciplinary engineering elective course and integrating these in research experiences for undergraduates (REU) and STEM education (GK-12) is discussed.




I. Introduction
In the digital generation, large volumes of data are collected in various forms in different endeavors related to business, science, engineering and biomedicine, among others. There is a need to make sense of the voluminous data for assessing the current status of the system and detecting an early indication of any possible deterioration of the system health. Computational intelligence (CI) techniques are ideal for such applications as tools of ‘knowledge discovery from data’ or in short, ‘data to knowledge’ for complex and often apparently intractable systems.


There is another kind of situation where the systems have to act proactively in view of the predicted system status in an unknown, uncertain and changing environment leading to development of ‘intelligent autonomous systems’. These systems form a broader class of newlycoined ‘cyber-physical systems’ or CPS. In a CPS, the ‘cyber’ resources representing computing, communication and control combine and coordinate  with ‘physical’ resources. For development of CPS systems, CI techniques are used with inspirations from nature. These systems have unique ability to learn and adapt to new situations utilizing the processes of generalization, abstraction and association with inspirations from nature [1-19].


There are main five paradigms of CI algorithms, namely,


(1) neural networks (NN),
(2) evolutionary computation (EC),
(3) swarm intelligence (SI),
(4) fuzzy systems (FS),
(5) immunological computation (IC).


The wide range of CI algorithms from these paradigms
include:
(1)- artificial neural networks (ANN);
(2)-genetic algorithms (GA), genetic programming (GP), differential evolution (DE); (3)- particle swarm optimization (PSO), ant colony optimization (ACO); (4) fuzzy inference system (FIS); and
(5)- artificial immune system (AIS), clonal selection algorithm (CSA). ANNs have been developed in form of parallel distributed network models based on biological learning process of the human brain(neuroscience)
[6]. GA and GP are developed as simulated evolution of ‘survival of the fittest’
(genetics) [1, 3].



Similarly, particle swam optimization (PSO) is proposed as a population based
stochastic optimization technique inspired by the social behavior of bird flocking [2, 4, 5, 9]. FIS embodies human reasoning and concept formation to deal with imprecise and uncertain information [7, 8]. AIS has been developed with an inspiration from the mechanisms of immune systems [10-13]. Application domains of CI include science, engineering, economics, social science, computing, bioinformatics and biomedicine, among others [4-7, 9, 12-15, 20-41]. There is a need to expose engineering students to CI and their multidisciplinary applications for better utilization of these techniques and their future development [16-19]. It is also important to introduce the CI techniques and their applications in the K-12 education through avenues of federal programs and initiatives (like REU and GK-12) for encouraging school students join STEM disciplines [42, 43].



This paper deals with an introduction of the CI techniques along with their several applications. In Section II, different CI paradigms are briefly discussed. Brief introductions to different hybridization schemes are given in Section III. Author’s experiences are presented covering several novel applications of combining CI techniques and utilizing the hybrid forms in different practical areas like engineering systems used in military and civilian applications, manufacturing, biomedical and health care systems as well as engineering education. In Section IV,the experiences of presenting a CI course and summer projects are also presented.
The importance of introducing the CI techniques and their multidisciplinary applications to engineering students as well as in K-12 STEM education is discussed in the concluding section.






II. Computational Intelligence (CI) Paradigms


In this section, the main paradigms of CI are briefly discussed for completeness. Some of the popular algorithms from each paradigm are considered here for lack of space. For details, readers are referred to texts [1-7, 13].


II.A Neural Networks (NN)


Artificial neural networks (ANNs) draw inspirations from neuroscience for generating applications of ANNs in data analysis, pattern recognition and control. Among different ANNs, multi-layer perceptron (MLP), radial basis function (RBF), and probabilistic neural networks (PNN) are most popular [4-6]. In general, an ANN consists of an input layer, an output layer and one or more hidden layers in between. The number of hidden layers, number of neurons in each layer, the activation functions for the neurons and the learning algorithm are some of the major issues to be considered in the implementation of an ANN. There are both supervised and unsupervised NN.



II.B Evolutionary Computation (EC)



Genetic algorithms (GA) and genetic programming (GP) are among the popular EC techniques used in a wide variety of applications. GAs represent a class of stochastic search procedures based on the principles of natural genetics and through simulated evolution process on a constant-size population of possible solutions in the search space. Each individual member of the population is represented by a string known as genome [1]. The genomes could be binary or realvalued numbers depending on the nature of the problem. The standard GA implementation involves the following issues: genome representation, creation of an initial population of individuals, fitness evaluation, selection of individuals, creation of new individuals using genetic operators like crossover and mutation, and specifying termination criteria. GP has a lot of similarities with GA. The main difference of GP and GA is in the representation of the solution. In the case of GA, the output is in the form of a string of numbers representing the solution. GP produces a computer program in the    form of a tree-based structure relating the different inputs (leaves) through mathematical functions (nodes) to the output (root node) [3].





II.C Swarm Intelligence (SI)



Particle swam optimization (PSO) is a population based stochastic optimization technique inspired by the social behavior of bird flocking [2]. PSO is a computationally simple algorithm based on group (swarm) behavior. The algorithm searches for an optimal value by sharing cognitive and social information among the individuals (particles). PSO has many advantages over evolutionary computation techniques like GA in terms of simpler implementation, faster convergence rate and fewer parameters to adjust [2, 4, 5]. The popularity of PSO is growing with applications in diverse fields of engineering, biomedical and social sciences, among others [9].



II.D Fuzzy System (FS)



Fuzzy logic (FL) has been used in many practical engineering situations because of its capability in dealing with imprecise and inexact information [7, 8]. The powerful aspect of fuzzy logic is that most of human reasoning and concept formation is translated into fuzzy rules. The combination of incomplete, imprecise information and the imprecise nature of the decisionmaking process make fuzzy logic very effective in modeling complex engineering, business, finance and management systems which are otherwise difficult to model. This approach incorporates imprecision and subjectivity in both model formulation and solution processes. The major issues involved in the application of FL or FIS are the selection of fuzzy membershipfunctions (MFs), in terms of number and type, designing the rule base simulating the decision process as well as the scaling factors used in fuzzification and defuzzification stages. These parameters and the structures are, in general, decided based on trial and error and expert knowledge.





II.E Immunological Computation (IC)



All living beings have the ability to present resistance and develop (partial or complete) immunity to disease-causing agents or infections. IC techniques have been developed with the ideas and metaphors from the biological immune systems [10-13]. IC techniques utilize the various aspects of the immune system like pattern matching, feature extraction, learning and memory, diversity, distributed processing, self-organization, and self-protection [13]. The development of an IC consists of three main stages: representation of the solution, evaluation of interactions, and procedure of adaptation. Various IC algorithms have been proposed based on model of adaptation, namely, bone marrow, negative selection, clonal selection, and continuous and discrete- immune network models [12, 13]. IC algorithms are finding applications ranging from biology to robotics.





III. Hybridization of CI




An important area of active research in CI is the hybridization of these techniques.This is often used to solve complex real-world problems where one technique is typically used to fix the weaknesses of the other. In adaptive neuro-fuzzy systems (ANFIS), the advantages of FL and ANNs are combined for adjusting the MFs, the rule base and related parameters to fit the training data set. The author has presented a large number of articles in the hybridization of CI techniques where the hybridization leads to much more effective algorithms. The author has combined ANNs (MLP, RBF, PNN), support vector machines (SVM), proximal SVM (PSVM), ANFIS
with GA, GP, both binary and real-valued PSO. The author has shown the applications of these hybrid CI techniques in the areas of machine condition monitoring, detection, diagnosis, prognostics [20, 23-29, 34-37]; intelligent manufacturing systems [30, 31, 33]; inventory control [21, 22]; biomedical applications and health care systems [32, 38, 39]. These hybrid CI techniques have been proposed for automatic selection of classifier (ANN, SVM, PSVM, ANFIS) structure and parameters, selection of significant system features from a pool, and selection of most important sensors (in the context of on-line condition monitoring and diagnostics) or sensor fusion. In the following subsections, some representative results from the author’s publications are briefly presented. Details can be obtained from the relevant articles.




III.A Machine condition monitoring, detection, diagnosis and prognostics [20, 23-29, 34-37]




Feature selection is an important issue in many real-world problems. Hybrid CI techniques have been proposed by author for feature selection in machine condition monitoring, detection, diagnosis and prognostics. Figures 1(a) and (b) show the role of a hybrid CI combination (PSO and PSVM) in separation of the data clusters for machine condition detection compared to principal component analysis (PCA). The classification success the CI (98-99%) is much than
PCA (59-65%) [35].




III.B Intelligent Manufacturing System [30, 31, 33]



Surface roughness is widely used as an index of product quality in finish machining processes. The requirement of desired surface roughness imposes critical constraints on selection of optimum machining parameters in engineering manufacturing processes. The selection of proper set of input features for accurate prediction of surface roughness is considered to be a challenge because of the uncertainty inherent in
the machining process. The author has shown applications of CI for
automated selection of the machining variables for modeling the surface
roughness.



III.D Biomedical Applications [32, 38, 39]





IV. Courses and Summer Projects [40, 41]


The author developed and offered a course on Engineering Applications of CI as a graduate course at Villanova. The students were introduced to different CI paradigms and their applications. The students applied the CI algorithms in their term projects. In addition, an exploratory project to provide research experiences on swarm robotics to high school studentswas initiated. A group of three simple mobile robots (Lego NXT) was used to study ‘search andrescue’ operation. PSO was used as the main algorithm.





V. Conclusions



The growing interest in education and research in the broad areas of cyber infrastructure, cyber systems for understanding of complex multidisciplinary systems and their development isevident from different programs adopted at the federal level (NSF, NIH, DoD). The need to educate and train the future generation in such ‘cyber’ related areas and CI cannot be overemphasized. It is imperative to attract more students to Science, Technology, Engineering and Mathematics (STEM) disciplines, in general, and to CI, in particular, with a transformative outlook in engineering programs. To start with, CI related courses should be introduced as senior level interdisciplinary senior elective in engineering. Several programs funded by NSF like REU and GK-12 should be actively considered to reach out to and involve school students (K-12), undergraduates and graduates in STEM disciplines.