Recently, a friend left a message saying, "I want to understand the key points of smoke testing, SIT, UAT, and regression testing in core system construction, how to design test cases, or related material recommendations."
This topic is quite broad. Testing includes not only business testing but also performance testing, security testing, etc. Different roles have different requirements for test cases. For example, bank business personnel like to write cases that run the entire transaction from start to finish, while testing company personnel like to write very detailed cases, etc. Regarding this, because I haven't undergone formal testing method training, I'll mainly share my personal understanding or feelings. Along the way, I've summarized some key basic knowledge and actual experiences of friends, sharing them so that more people can find direction.
- Target audience:
Bank employees, business personnel, testers.
- Problems solved by this article:
For testers new to banking business, learning will help systematically understand bank system testing knowledge, aiding future work.
- This article is divided into four parts:
- Main tasks of bank testing
Classification and basis of bank testing
Test case design for bank testing
Bank test execution requirements and standards
1 Main tasks of bank testing
As everyone's financial advisor, banks are very sensitive to money. Frequent or even occasional software failures can shake customer confidence. If a hacker attack occurs and personal property is threatened, the bank will inevitably suffer losses. Therefore, banks have very high quality requirements for systems, pursuing functional stability, reliable performance, high security, and ultimately achieving customer trust, ensuring the safety of bank and personal property.
The prerequisite for ensuring high system quality is testing. Testing is a very important phase in the entire core project, so the role of testers is crucial. Let's start with the main tasks of the testing phase.
(1) Test rule writing
Test rules are derived from requirement analysis. They are the key points that need to be tested after analyzing the original requirements, and are the first step in testing work. Generally, intermediate and senior testers write test rules. The more detailed and precise they are, the better understanding of the tested business, making it easier to discover system and business issues, and better control test quality and progress.
If the test requirement analysis is unclear, the key points of test rules will be unclear, and the preparation of test cases will be even more baseless. This may cause waste of time or resources, inaccurate estimation of test workload, leading to project delays. So, how to improve requirement analysis ability?
First, by reading requirement documents to understand the background of requirements, the purpose and user scenarios. Note down any doubts encountered, communicate more with business to quickly familiarize with business knowledge. Second, analyze the rationality of requirements from the user or business perspective. Provide design suggestions to requirements or developers to avoid being bound by inertia. Finally, make full use of learning resources around you or online, such as good blogs or public accounts, learn from predecessors' experiences and apply them to actual work.
Let's return to the subtitle. Regarding test rule writing, for junior testers, initially imitate experienced people's cases. Then through more learning, thinking, summarizing, and sharing, requirement analysis ability will greatly improve, and eventually you can write test rules smoothly.
Test documents don't need to be too complicated. You can just use Excel. Take the regular partial withdrawal transaction of the core system deposit module as an example, see the picture below:
(2) Test case writing
Early on, while developers are designing and coding, testers are continuously refining and adjusting test plans and completing test case writing. Test case writing is actually based on the above test rules, refining specific test cases, including test objectives, test environment, input data, test steps, expected results, etc.
But the degree of refinement of test points is a matter of balance. We need to grasp the degree. Too coarse test points have no guiding significance, too fine test points may make us focus too much on details and ignore overall testing, also not achieving a guiding effect. So we must grasp the degree of refinement. As a beginner, what problems might you encounter?
For example, many people don't know how to start writing test cases, but if they hesitate to write, they worry about affecting the overall test schedule. For such a hesitant mindset, the suggestion is to not worry about whether your cases are good or bad, just write them down. Or refer to previously written public test cases, even directly quote them. This can avoid unnecessary time waste, but reference is not copying. While quoting, be sure to think about the specific test points of this requirement.
Second, test cases will go through case review, with senior testers and business personnel checking. Problems in test cases will be found, and reviewers will give modification suggestions to each person. So, rest assured and write down the test cases you think of. This helps find problems and solve them better.
Also, no one's test cases can be said to be perfect and comprehensive. Through continuous review, try to be as comprehensive as possible and achieve high coverage. However, experienced employees have more experience and insight than novices, so they definitely have a set of appropriate methods when writing test cases. Next, take opening a time deposit (lump sum) via mobile banking as an example to share some dry goods, so that all testers have the means to cook.
(3) Test case review
After test case writing is completed, the test manager will organize a test case review, i.e., check the test cases. This is generally done before developers submit the transaction or function for testing. Key stakeholders from the bank's business or technology departments should participate in the review, going through cases one by one to reconfirm everyone's understanding of the requirements. Test case review is a crucial part of the testing process. Why is case review so important?
First, through test case review, not only can deviations in understanding of requirements among product, development, and testing be eliminated, but also consistency between test executors and test case designers in terms of test quality assurance can be ensured.
Second, through test case review, product and development can evaluate the rationality and comprehensiveness of cases, point out unreasonable or missing parts, so as to better improve test cases and improve their quality.
Furthermore, if development cannot hold technical review meetings due to various constraints, then in case review you can also communicate with development to confirm implementation methods, pay attention to test points of different implementation methods to achieve comprehensive testing.
Finally, as the saying goes, testers are the front lights of the project, exploring the way. A good doctor treats diseases before they occur. Testers should dig out potential pitfalls in the early stages of the project and remind developers to be careful to avoid falling into pitfalls. At the same time, this reduces the probability of bugs and saves development and testing costs.
Therefore, because many requirement details cannot be fully considered in the requirements phase, repeated communication with relevant personnel is used to continuously refine. Generally, this review will be repeated about three times to improve the cases. Later, due to tight project schedules or too frequent requirement changes, selective quick reviews of cases are often conducted.
(4) Smoke testing
After the test case review passes and the development submits for testing, testers quickly complete a round of "smoke testing". The purpose of smoke testing is to confirm that basic functions are normal and that subsequent formal testing can proceed, minimizing unknown risks of formal testing and preventing bug blocks from delaying the test schedule. However, some projects start smoke testing halfway through the review, reviewing and smoking simultaneously.
From the perspective of the core development team, before notifying testers to conduct smoke testing, the development team usually does some transaction verification internally. Especially during the migration smoke test phase, leaders from all sides pay special attention because problems in migration smoke directly affect the start time of UAT or whether it can be launched on schedule. Therefore, basically if problems are found, they require immediate resolution and verification, or resolution within the day. Project assistants will follow up, confirm one by one, collect feedback, etc.
Additionally, regarding the construction of smoke test cases, two suggestions: First, the test case administrator communicates with the development manager to confirm new function points; second, determine which existing cases are still valid for the new project, delete cases that are no longer applicable, thus establishing a new set of test cases.
(5) Functional review
While testers start executing test cases, business personnel will organize a "functional review meeting" or "demo meeting". Using the test environment, they display usable functions to relevant stakeholders as early as possible to further ensure that what is built is what everyone wants.
(6) Testing
Next, testers will conduct multiple rounds of testing, a cycle of "find bugs, developers fix, retest, find new bugs". From the second round onward, it can be called "regression testing". After multiple rounds of testing, the project will require various user representatives from the bank to conduct more detailed UAT.
Generally, at the end of SIT or the beginning of UAT, it is the time when defects are most numerous, and also the hardest time for developers (personal feeling, not sure what testers experience at this stage).
During this period, various problems arise, such as parameter inconsistencies, functions still not matching requirements after repeated modifications, incorrect print output fields, version management issues leading to retest failure of previously successful cases, fixing one bug causing new bugs, resolution time exceeding deadlines, various errors in nightly batch processing, people urging progress, etc. It can be overwhelming.
Actually, the more such times, the less you can panic. Stay calm. Even huge bugs must be handled one by one. Adjust your personal state and working methods. Once you get through this period, things will be much better. After multiple rounds of UAT testing and all bugs are properly handled, it enters the UAT wrap-up, program version freeze, parameter check and freeze, drills and launch preparation.
At this time, commercial preparations have already started. Business personnel may prepare user-facing function introductions, selling point documents, product update announcements; train service and sales personnel; operations personnel may be planning promotion plans; sales personnel may be updating sales scripts... Multiple departments collaborate, giving a feeling of fighting together.
★ Glossary
Smoke Test: This test is short, just enough time for a smoke. Some also compare it to checking basic functions of a new circuit board. After soldering any new circuit board, power it on first. If there is a design defect, the circuit board may short circuit and smoke.
UAT (User Acceptance Test): User acceptance test. Of course, a better approach is to directly let users test.
Acceptance Test: In addition to testing all system functions and performance at a high level, it also requires checking project deliverables, such as project phase documents, user manuals, etc., for completeness and compliance with specifications.
Regression Test: After deploying a modified code version, retest previous bugs and verify the correctness of the version. Often a system go-live requires multiple regressions. Some companies adopt multiple rounds (first, second, third, etc.) as forms of regression testing, but each round has a different focus.
Bug: Refers to defect or failure. The difference is that those found before the project goes live are called defects, and those found after go-live are called failures. Failures often harm users. Teams also establish classification systems for failures and corresponding penalties for responsible persons.
2 Classification and basis of bank testing
In the computer industry, developers have their main areas of involvement in actual development work, such as COBOL, Java, Python, PHP, C, etc. The same applies to testers. Therefore, bank testing has many classifications. According to test content, it can be divided into: functional testing, performance testing, security testing, and other types.
(1) Functional testing
Functional testing can be divided into module functional testing, business functional testing, scenario functional testing, and message functional testing. Continue with the example of mobile banking lump sum deposit function:
Module functional testing: e.g., CRUD, dropdown selection, value field input, reaction after button click; Business functional testing: e.g., time deposit to current deposit function test; Scenario functional testing: e.g., time deposit process, early account closure, early partial withdrawal, linking business functions into a chain; Message functional testing: e.g., interaction message testing with payment system or core system.
(2) Performance testing
Performance testing can be divided into large volume scenario testing, end-to-end concurrency testing, test with stubs, business stress testing.
(3) Security testing
Security testing can be divided into message encryption testing, password security testing, penetration testing (firewall), channel transmission security testing.
(4) Other types
Other types include system testing, hardware testing (POS machines, smart counters), peripheral testing (ATMs).
Next, let's talk about the basis of testing.
The basis of testing can be divided into six points:
Bank business rules, e.g., "Relevant regulations on Chinese uppercase in bank checks".
Business scenario requirements, e.g., transfer business scenario.
Accounting bookkeeping rules, e.g., debit-credit bookkeeping method.
Various documents issued by the central bank, e.g., PBOC's "Notice on Implementing Individual Account Classification Management System".
Input from various requirement documents, e.g., time deposit function specification.
Others, e.g., system prototypes.
3 Test case design for bank testing
(1) Case design classification
(2) Requirements and standards
(3) Precautions
4 Bank test execution requirements and standards
(1) Test execution requirements and standards
Execution must strictly follow business scenarios and business processes.
The test data used must be reliable and complete.
Test execution result data must be correctly stored and correctly calculated.
After execution, pay special attention to verification and retention of evidence traces.
During test execution, you must consider actual operation scenarios of different users, and must involve control situations of different institutions, positions, passwords, etc., in permission control.
(2) Execution precautions
Strictly follow the cases to ensure consistency between test and case content.
Test data must be reliable and valid data generated according to business processes, not arbitrary data manually added.
Batch transaction focus on state changes, calculation changes, and migration correctness of processed bulk data.
Pay special attention to inconsistencies with expected results in the cases.
Arrange cross-testing as much as possible.
Conclusion
Previously, some friends asked about switching from testing to development or requirements roles. I think it mainly depends on what you are good at or like. For example, if you are good at communication or coordination, then doing requirements might be better. If you like technical work, switching to development allows you to better flex your skills. If you are good at reverse thinking, good at discovering anomalies and branches outside the main line, then testing is very suitable. That's all for the topic of testing. Interested friends can follow up, and there may be opportunities to expand.
References and notes:
"Summary of Test Case Design"
"Summary and Reflection on Functional Testing from a Test Expert"
"ATesting Sharing Session - Banking Industry Testing"
Author: Dai Tangming
Reprinted from WeChat public account: Xiao Dai De Ba De
Original link: https://mp.weixin.qq.com/s/kwfJXT9IjIOM-cKOxyNNuw