What is Performance Engineering?

Performance Engineering is end-to-end optimization by providing constant monitoring and testing, unlike Performance Testing which traditionally occurs as a part of quality assurance after the software development process is over.

The main goal is to provide better performance to user by discovering potential issues as early design stage and avoid cost of fixing last-minute bottlenecks.

Difference between Performance Testing and Performance Engineering

Performance Testing	Performance Engineering
Applied load to application and test how it performs during heavy load	Designing application keeping performance metrics in mind
QA process which is conducted once development is over	Starts from design phase and occurs through all development and testing phases
Conducted to identify and raise defects	Conducted to identify bottleneck root cause and fixing
Application design and architecture knowledge not required for performance testing	Application design and architecture knowledge required for performance engineering

Performance Engineering Life Cycle

Performance Engineer first need to collect non-functional requirements from customer. For example, what is the expected user load they are looking for, expected response time for page and what type of performance tests can be included and what is the purpose.

After collecting requirements, performance test strategy is required to test business requirement and successfully deliver product to end user.

Performance Engineer need to design performance test scenarios that consider user variability, test data and target metrics and scripts for end-to-end flow.

Performance test execution refers to run the tests specific to performance testing like load test, soak test, stress test, spike test etc. using performance testing tool.

After execution, test results analysis needs to be done and report need to be prepared.

Performance benchmarking is the process of measuring and analyzing an organization’s performance of products, services, operations, and other business processes against competitors.

Top Trends in Performance Engineering

• Cloud based infrastructure and architecture
• Distributed and third-party services
• Cloud Load Testing
• Network Virtualization: Test, Tune, Benchmark
• Stitching Metrics and Logs
• Machine Learning: Enabling continuous performance engineering in production environment
• Real-User Monitoring Capability
• Agile/ Iterative development

Tools and Utilities

• LoadRunner, JMeter, Grinder, Webload, OpenSTA
• Web servers like Apache, IIS
• Application servers like Apache Tomcat, Jboss, weblogic, websphere
• Databases like Oracle, My SQL, PostgreSQL

Challenges in Performance Engineering

• Lack of understanding of concepts
• Lack of willingness to invest in performance engineering tasks
• Lack of skills
• More focus on functional requirements and application design at initial stages
• Choosing right tools as there are multiple tools available in market
• Practical resource limitations on number of users
• Exploding user population
• Constant cost and budget pressure
• Accurate measurements of response times
• Generation and analysis of reports

Best Practices

• Companies prefer performance tuning to correct performance related issues as they think its cost effective, but it’s not the case, it might save time, but it’s not effective strategy for long run. Refactoring should be done rather than tuning
• Each test should be run multiple times in same environment, if test produces same results after multiple tests, better performance testing can be done by changing certain variables
• Single user load test needs to be validated, every user should run different test script at same time and validate minimum response time for single-user load, these results can be used to identify minimum and maximum response time

Benefits of Performance Engineering over Performance Testing

• Test coverage is good for performance engineering projects as we get more time to work on initial level, i.e., Requirement analysis, whereas, in performance testing, we have limited scenarios for validation due to time and budget limits
• Performance engineering projects can be released faster as issues can be detected and fixed in early stages compared to performance testing where testing is performed at end of release and issue detection can cause delay to schedule
• Utilization is high, as involved team members work in continuous collaboration. For performance testing, different teams work on various tasks and do not collaborate constantly
• Follows proactive approach as its performed parallelly with development and testing, whereas performance testing follows reactive approach as it is performed after other levels of testing
• No dedicated time required for performance engineering, however, for performance testing, dedicated time would be needed for application KT as no/ little prior discussion between teams

Conclusion

Companies adopt performance testing to ensure application speed and stability with new release, however, this approach has few challenges. Performance engineering is about providing solution to avoid last minute bottlenecks reducing time and budget with quality. Companies must accept performance engineering to stay one step ahead.