The pervasive cyber threat to DoDs, public sector and industry means cyber vulnerability and penetration assessment (CVPA) and testing is no longer an option but rather about managing an acceptable risk of how much testing is enough (Christensen, 2017 & 2015).
The pervasive cyber threat to DoDs, public sector and industry means cyber vulnerability and penetration assessment (CVPA) and testing is no longer an option but rather about managing an acceptable risk of how much testing is enough (Christensen, 2017 & 2015). Not testing, simply means not knowing, and thus an unassessed risk, while not re-testing fielded systems at some interval means an atrophy of security confidence at an unknown rate (Brown et al., 2015). In critical systems, operators are mounting continuous defensive cyber operations, sometimes extending to supply-chain monitoring through-life (Fowler et al., 2017; Alberts et al., 2017), but in Australia these precautions are largely only on live networked systems (Joiner, 2017). Outside the U.S. and particularly the U.S. DoD, there is still limited understanding of the risk of cyber threats to software-intensive systems which are only occasionally updated or networked, mainly because of the prevailing view of cyber threat as an internet hacker seeking protected information. Public sector and financial systems are vulnerable to more sophisticated probing and logic disruptions that can be electromagnetic lodged at low power with no connectivity, leading to denial of service or false service (Joiner & Rehmann, 2017). Without CVPA and some defensive posturing even for fielded legacy systems, significant risk exists that at a time of a potential enemy or criminal entity's choosing, systems will be denied or interfered without detection for an unknown period of malicious intrusion (Christensen, 2017).
Forming a CVPA test capability is dramatically easier if the other ICT test capabilities (i.e., usability, integration & performance) are robust and appropriately part of project governance and a benefits realisation decision-making culture. Inevitably in capped schedules and budgets, increased cyber-resilience involves trade-offs: (1)with user requirements, such as determining through structured test what users value more; (2)with integration, such as limiting connectivity to limit cyber-threat exposure; or (3)with performance, such as increasing the threat detection algorithms and reducing system processing for main functions. The test design, test analysis skills and test infrastructure required to manage CVPA testing are, with only a few key additions, supported by the test skills and test infrastructure of the other ICT test types. For example, industries, public sector and DoDs that have invested in software integration laboratories, software system support centres, or Live Virtual Constructive (LVC) test networks can adapt these to allow for multi-security CVPA testing €” in essence extending integration and capability upgrade infrastructure to be cyber ranges that can concomitantly manage evolving cyber threats and deliver greater cyber resilence. If such infrastructure has been outsourced and is proprietary, then contractual changes will be needed to safeguard connection to government-managed representative cyber threats. Similar to test infrastructure, additional test design skills can be added to integration and performance testers to manage the additional rigor which is usually necessary for cyber resilience. Combinatorial test design has been instrumental in achieving greater cyber resilence with three-way through to six-way combinatorial test rigor being achieved, often while deriving new efficiencies (Kuhn et al., 2016) and other defect-protection rigor. To see how this is possible, examine the industry six-sigma software testing award overview by Mackertich et al. (2017). Joiner (2018) has adapted test design education to give early awareness of these additional cybersecurity test techniques as first released in freeware by Kuhn et al. (2010), but also used by big software industries (Tatsumi, 2013).Industry and departments have been slow to adopt another protective process layer such as that outlined by the U.S. DoD (2015) and Brown et al. (2015), which has led industry bodies to develop minimum additional cyber planning and testing checks to overlay standard systems engineering (Nejib et al., 2017; Mead & Woody, 2017). These process links and explanations offer the greatest promise to normalise cybersecurity in industry, at least in the U.S.. Probably the last and most difficult extension for CVPA testing from hitherto ICT testing is the skill of defensive (blue) and penetration (red) teams war-gaming the cyber threat as described well by Christensen (2017). These are military skills applied in a new domain and unfortunately necessary for public sector and critical industries to adopt if they are to be reasonably defensive to malicious threats. Legal protections in cyber are a long way from being instituted (Austin, 2016; Heinl, 2016) and deterrance is very difficult whilst ever attribution is so hard. Even if legal recourses become viable, public sector and industry war-gaming is necessary at some level for the defensive capability to exist to collect evidence for legal attribution.
This project aims to document the many cybersecurity test design strategies and analysis techniques that have evolved such as the combinatorial rigor techniques, the investigative fuzz testing, and mixed versions of the two. The project will explain how these are being used as test design strategies by public sector and critical industries for cyber-resilient systems and then do comparitive analyses of the efficacy and efficiency of the test strategies in different industry scenarios so as develop clear guidance, including for the infrastructure and skills of the testers. The project will deliver a textbook to help the cybersecurity industry link the rapidly evolving test design strategies and techniques in this field to education of urgently required cybersecurity test practitioners (Henry, 2017). Ideally, the project once established will be supported by public sector or industry from whence the PhD student comes, including trialing and refining the guidance with postgraduate coursework practitioners.
Contact:
Dr Keith Joiner,
CSC
School of Engineering & IT
Intelligent Security
