The growth of Cyber-physical Systems (CPS) has been increased in recent years. This has led to the coupling of highly complex cyber-physical components. With the integration of such complex components, new security challenges have emerged. Studies involving security issues in CPS have been quite difficult to be generalized due to the presence of heterogeneity and the diversity of the CPS components. These systems are subject to various vulnerabilities, threats and attacks, as a consequence of complex versions of CPS being introduced over time. This paper deals with vulnerabilities caused due to improper configurations in the software component of cyber-physical systems. Evolutionary algorithms such as Genetic Algorithms (GA) and Particle Swarm Optimization (PSO) can be employed to adequately test the underlying software for certain categories of vulnerabilities. This paper provides a detailed sensitivity analysis of these evolutionary algorithms in order to find out whether changing parameters involved in tuning these algorithms affect the overall performance. This analysis is based on the estimate of the number of generation of secure vulnerability pattern vectors under the variation of different parameters. The results indicate that while there is no evidence of influential parameters in Genetic Algorithms (i.e., mutation rate and population size), changes in the parameters involved in Particle Swarm Optimization algorithms (i.e., velocity rate and fitness range) have some positive impacts on the number of secure configurations generated.