軟件工程理論與實踐課件：第9章 系統(tǒng)的測試

1、Testing the System系統(tǒng)的測試,SOFTWARE ENGINEERING,9.1principles of system testing系統(tǒng)測試的原則,The objective of unit and integration testing was to ensure that the code implemented the design properly. The objective of system testing was to ensure that the system does what the customer wants it to do.單元測試和集成測試

2、的目標(biāo)是保證編碼正確地實現(xiàn)了設(shè)計。系統(tǒng)測試的目的是要保證系統(tǒng)做顧客希望它做的事情。 To understand how to meet this objective, we first must understand where faults in the system come from.為了理解怎樣實現(xiàn)這個目標(biāo)，首先必須理解系統(tǒng)中的錯誤來自何處。,一、Sources of Software Fault軟件錯誤的根源,Incorrect, missing or unclear requirements Incorrect or unclear translation Incorrect or

3、 unclear design specification Misinterpretation of system design Misinterpretation of program design Incorrect documentation Incorrect syntax or semantics Incomplete test procedures New faults introduced when old ones corrected Incomplete test procedures Incorrect user documentation Poor human facto

4、rs Changes in requirements,Requirements analysis,System design,Program design,coding,Unit/integration test,System test,Maintenance,一、Sources of Software Faults軟件錯誤的根源,不正確、遺漏或表述不清楚的需求 不正確、遺漏或表述不清楚的轉(zhuǎn)換 不正確或不清楚的設(shè)計說明 系統(tǒng)設(shè)計的錯誤解釋 程序設(shè)計的錯誤解釋 不正確的文檔 不正確的語法或語義 不完整的測試過程 當(dāng)糾正舊的錯誤時引入的新錯誤 不完整的測試過程 不正確的用戶文檔 人為因素 需求變動

5、,需求 分析,系統(tǒng)設(shè)計,程序設(shè)計,編碼（程序?qū)崿F(xiàn)）,單元/集成 測試,系統(tǒng)測試,維護,二、System testing process 系統(tǒng)測試過程,There are several steps in testing a system(測試一個系統(tǒng)時有如下幾個步驟)： Function testing(功能測試): does the integrated system perform as promised by the requirements specification.系統(tǒng)能按需求規(guī)格說明的要求運行嗎? Performance testing(性能測試): are the non-fu

6、nctional requirements met?是需求說明中的非功能需求? Acceptance testing(驗收測試): is the system what the customer expects?是客戶所希望的系統(tǒng)? Installation testing(安裝測試): does the system run at the customer sites?系統(tǒng)能運行在客戶的環(huán)境嗎?,System Testing Process,Installation test,Performance test,Function test,Acceptance test,System func

7、tional requirements,Other software requirement,Customer requirement specification,User environment,Integrated modules,Functioning system,Verified, validated software,Accepted system,System in use !,spin plan for gradual testing螺旋化漸進測試計劃,Large systems are sometimes unwieldy when tested as one enormou

8、s collection of components. A system can be viewed as a nested set of levels or subsystems. Each level is responsible for performing at least the functions of those subsystems it contains. Similarly, we can divide the test system into a nested sequence of subsystem and perform the system test on one

9、 subsystem at a time. The incremental testing has made fault detection and correction much easier. incremental testing requires careful planning. The test team must create a build plan to define the subsystems to be tested. A level or subsystem of a build plan is called a spin. The spins are number,

10、 with the lowest level called spin zero. The build plan describes each spin by number, functional content, and testing schedule. 在把大型系統(tǒng)作為一個巨大的組件集合進行測試時，有時會很不靈活。一個系統(tǒng)可以看成是各個層次或子系統(tǒng)的嵌套集合。每一層至少負(fù)責(zé)執(zhí)行它包含的那些子系統(tǒng)的功能。同樣可以把測試系統(tǒng)劃分成子系統(tǒng)的嵌套序列，然后每次只對一個子系統(tǒng)進行系統(tǒng)測試。這種遞增的測試使得錯誤檢測和糾正更加容易。遞增的測試需要仔細(xì)計劃。測試小組必須創(chuàng)建一個構(gòu)建計劃以定義要測試的子系

11、統(tǒng)。構(gòu)建計劃的一層或子系統(tǒng)被稱為一個螺旋層。螺旋層有編號，最低的一層稱為螺旋層0。構(gòu)建計劃描述了每一個螺旋層、功能的內(nèi)容和測試進度。,三、Configuration management配置管理,System testing must also take into account the several different system configurations that are being developed. A system configuration is a collection of system components delivered to a particular cus

12、tomer. Developing and testing these different configurations requires configuration management, the control of system different to minimize risk and error . 系統(tǒng)測試還必須考慮正在開發(fā)的幾種不同的系統(tǒng)配置。系統(tǒng)配置是交付給某個特定顧客的系統(tǒng)組件集。開發(fā)和測試這些不同的配置需要進行配置管理，對系統(tǒng)差別進行控制以盡量減少風(fēng)險和錯誤。,versions and releases版本和發(fā)布,A configuration for a particu

13、lar system is sometimes called a version. A new release of the software is an improved system intended to replace the old one. The configuration management team is responsible for assuring that each version or release is correct and stable before it is released for use, and that changes are made acc

14、urately and promptly. 特定系統(tǒng)的一個配置稱為一個版本。一個新的軟件發(fā)布（版本）是一個改進了的系統(tǒng)，旨在替代老的系統(tǒng)。配置管理小組負(fù)責(zé)確保在發(fā)布的產(chǎn)品投入使用之前，每個版本或發(fā)布是正確且穩(wěn)定的、變動進行是正確且迅速的。,Regression testing 回歸測試,Regression testing identifies new faults that may have been introduced as current ones are being corrected. A regression test is a test applied to a new ver

15、sion or release to verify that it still performs the same functions in the same manner as an older version or release. if the team is following a policy of strict regression testing, the testing involves these steps: 回歸測試用來確定在糾正當(dāng)前錯誤的同時可能引入的新錯誤。回歸測試是適用于新版本或新發(fā)布的一種測試，用以驗證它仍然以同樣的方式執(zhí)行與舊版本或發(fā)布一樣的功能。如果測試小組遵

16、循嚴(yán)格的回歸測試策略，則測試應(yīng)該包括這些步驟： Inserting your new code 插入新代碼。 Testing functions known to be affected by the new code 測試已知的被新代碼影響的功能。 Testing essential functions of M to verify that still work properly 測試M的基本功能，以驗證它們?nèi)耘f正確的工作。 Continuing function testing of M+1 繼續(xù)M+1的功能測試。,deltas, separate files and condition

17、al compilation差別文件、單獨文件和條件編譯,There are three primary ways to control versions and release, and each has implications for managing configurations during testing .控制版本和發(fā)布有三種方式，每一種都包含測試中的配置管理。 Keep separate files for each different version or release. 對每個不同的版本或發(fā)布保留單獨文件。 Designate a particular version t

18、o be the main version, and define all other versions to be variations from the main. We need store only the differences, rather than all the components, for each of the other versions. The difference file called delta, contains editing commands that describe how the main version is to be transformed

19、 to a different version. 指定一個特殊的版本為主版本，而定義其他所有版本是主版本的變種。只需存儲所有其他版本與主版本之間的差別，而不是所有組件。差別文件稱為delta,包含了描述主版本怎樣變換到不同版本的編輯命令。 Controlling file difference is to use conditional compilation. 使用條件編譯來控制文件差別。,change control 變動控制,The configuration management team works closely with the test team to control all

20、aspects of testing. Any change proposed to any part of the system is approved first by the configuration management team. The change is entered inn all appropriate components and documentation, and the team notifies all who may be affected. 配置管理小組與測試小組緊密合作，以控制測試的所有方面。對系統(tǒng)的任何一部分的改動建議首先都要得到配置管理小組的批準(zhǔn)。改動

21、進入適當(dāng)?shù)慕M件和文檔中，而配置管理小組通知改動可能影響到的所有人員。,四、Test team 測試小組,Professional testers（ ）: organize and run the tests Analysts（ ）: who created requirements System designers（ ）: understand the proposed solution Configuration management specialists （ ）: to help control fixes Users（ ）: to evaluate issues that arise,

22、9.2 Function Testing 功能測試,System test begins with function testing, and focuses on functionality. Function testing is based on the systems functional requirements. 系統(tǒng)測試始于功能測試。重點是功能。功能測試的基礎(chǔ)是系統(tǒng)的功能性需求。,一、purpose and roles 目標(biāo)與任務(wù),Each function can be associated with those system components that accomplis

23、h it. For some functions, the parts may comprise the entire system. The set of actions associated with a function is called a thread, so function testing is sometimes called thread testing. 每個功能都與完成它的那些系統(tǒng)組件有關(guān)。對于有些功能，這些部分可能包含了整個系統(tǒng)。與功能相關(guān)的活動集合稱為線程，因此功能測試有時稱為線程測試。 Function testing compares the system s

24、actual performance with its requirements, so the test cases for function testing are developed from the requirements document. 功能測試將系統(tǒng)的實際性能與其需求進行比較，因此功能測試實例是從需求文檔中發(fā)展而來的。,Guidelines for function testing功能測試的指導(dǎo)方針,Have a high probability of detecting a fault 以高概率檢測出錯誤。 Use a test team independent of de

25、signers and programmers 使用與設(shè)計人員和編程人員無關(guān)的測試小組。 Know the expected actions and output 知道期望的活動和輸出。 Test both valid and invalid input 測試合法和不合法輸入 Never modify the system just to make testing easier 不會為了測試容易而去修改系統(tǒng)。 Have stopping criteria 具有停止標(biāo)準(zhǔn)。,二、Cause-and-Effect Graphs 因果圖,Describe logical relationships b

26、etween inputs and outputs or between inputs and transformations. Cause input left node effect output right node,描述輸入與輸出之間或輸入與轉(zhuǎn)化之間的邏輯關(guān)系,Notation（因果圖符號）,因果圖示例,水文監(jiān)控系統(tǒng)中某一功能的需求： The system sends a message to the dam operator about the safety of the lake level.系統(tǒng)向大壩操作人員發(fā)送有關(guān)湖泊水位安全性的消息 Input: 函數(shù)LEVEL(A, B)

27、A為壩上游水位；B為24小時內(nèi)的降雨量 Processing: 計算函數(shù)LEVEL(A, B)的值 Output :顯示下列信息之一 “LEVEL = SAFE” “LEVEL = HIGH” “INVALID SYNTAX”,因果圖示例,Separate the requirement into five “cause”這些需求分成5種“原因” 函數(shù)名為 “LEVEL” 參數(shù)體“(A, B)” 函數(shù)值= LOW 函數(shù)值= SAFE 函數(shù)值= HIGH Three “effects”三種“結(jié)果” 顯示 “LEVEL = SAFE” 顯示 “LEVEL = HIGH” 顯示 “INVALID S

28、YNTAX” 為函數(shù)體的語法檢查增加兩個中間節(jié)點 The command is syntactically valid命令是合法的 The operands are syntactically valid操作數(shù)是合法的,因果圖示例,3,E3,4,E1,5,E2,函數(shù)名是否為 “LEVE” 參數(shù)體“(A, B)” 函數(shù)值= LOW 函數(shù)值= SAFE 函數(shù)值= HIGH,E1:顯示 “LEVEL = SAFE” E2:顯示 “LEVEL = HIGH” E3:顯示 “INVALID SYNTAX”,操作數(shù)是合法的 命令是合法的,因果圖的判定表 Table 9.2. Decision table

29、for cause-and-effect graph.,I - cause is invoked or true原因被調(diào)用或為真 S cause is suppressed or false原因被禁止或為假 A: Absent（結(jié)果）未出現(xiàn); P : Present （結(jié)果）出現(xiàn) 25= 32, however, with the help of cause-and-effect graph, we reduces the number.,9.3 Performance tests性能測試,Performance testing addresses the nonfunctional requ

30、irements. System performance is measured against the performance objectives set by the customer as expressed in the nonfunctional requirements. Performance testing is designed and administered by the test team, and the results are provided to the customer. Because performance testing usually involve

31、s hardware as well as software, hardware engineers may be part of the 性能測試針對的是非功能性需求。系統(tǒng)性能是根據(jù)顧客在描述非功能性需求時設(shè)置的性能目標(biāo)來度量的。性能測試由測試小組進行設(shè)計和管理，結(jié)果提供給顧客。因為性能測試通常涉及硬件和軟件，硬件工程師會是測試小組的部分成員。,Types of tests 測試類型,Stress tests Volume tests Configuration tests Compatibility tests Regression tests Security tests Timing

32、tests,Environmental tests Quality tests Recovery tests Maintenance tests Documentation tests Human factors (usability) tests,Stress test 強度測試：evaluate the system when stressed to its limits over a short period of time. 評價系統(tǒng)在短時間內(nèi)到達極限時的表現(xiàn)。 Volume test 容量測試：address the handling of large amounts of data

33、 in the system.針對的是系統(tǒng)中大量數(shù)據(jù)的處理。 Configuration test 配置測試：analyze the various software and hardware configurations specified in the requirements.分析需求中指定的不同軟件和硬件配置。,Compatibility test 兼容性測試： when a system interfaces with other systems. We find out whether the interface functions perform according to the

34、 requirements.當(dāng)一個系統(tǒng)與另一個系統(tǒng)交互時，檢查接口功能是否按照需求的要求執(zhí)行。 Regression test 回歸測試：when the system being tested is replacing an existing system. The regression tests guarantee that the new systems performance is at least as good as that of the old.當(dāng)待測系統(tǒng)替代了一個現(xiàn)有系統(tǒng)時，回歸測試保證新系統(tǒng)的性能至少和老系統(tǒng)一樣。 Security test 安全性測試：ensure t

35、hat the security requirements are met.確保滿足安全性需求。,Timing test 計時測試：if a transaction must take place within a specified time, the test performs that transaction and verifies that the requirements are met.如果一個事務(wù)必須在指定時間內(nèi)發(fā)生，測試執(zhí)行這個事務(wù)，驗證是否滿足需求。 Environmental test 環(huán)境測試：look at the systems ability to perform

36、 at the installation site. 了解系統(tǒng)在安裝地點的執(zhí)行能力。 Quality test 質(zhì)量測試：evaluate the systems reliability, maintainability, and avilability.評價系統(tǒng)的穩(wěn)定性、可維護性和可用性。,Recovery test 恢復(fù)測試：address response to the presence of faults or to the loss of data, power, devices, or services. 針對系統(tǒng)對出現(xiàn)錯誤或丟失數(shù)據(jù)、電源、設(shè)備或服務(wù)時的反應(yīng)。 Documenta

37、tion test 文檔測試：ensure that we have written the required documents.確保我們已經(jīng)編寫了所需的文檔。 Human factors （usability ） test人為因素（使用性）測試：investigate requirements dealing with the user interface to the system.調(diào)查與系統(tǒng)的用戶接口有關(guān)的需求。,9.4 Reliability, Availability, and maintainability 可靠性、可用性和可維護性,Software Reliability (R

38、) Probability that the system will operate without failure for a given period of time Software Availability (A) Probability that a system is operating successfully at a given point in time Software Maintainability (M) Probability that a maintenance activity can be carried out within a stated time in

39、terval,Measuring Reliability Availability, and maintainability度量可靠性、可用性和可維護性,MTTF (Mean Time to Failure) MTTR (Mean Time to Repair) MTBF (Mean Time between Failures) MTBF=MTTF+MTTR R=MTTF/(1+MTTF) A=MTBF/(1+MTBF) M=1/(1+MTTR),9.5 Acceptance tests 驗收測試,Purpose and roles 目的和任務(wù) The purpose of acceptanc

40、e testing is to enable the customers and users to determine if the system we built really meets their needs and expectations. thus, acceptance test are written, conducted, and evaluated by the customers. 驗收測試的目的是使顧客和用戶能判定我們構(gòu)建的系統(tǒng)是否真正滿足了他們的需要和期望。因此驗收測試的編寫、實施和評價都是由顧客來進行的。,Types of acceptance tests驗收測試的

41、類型,Benchmark test（ ）： Pilot test（ ）: install on experimental basis Alpha test: in-house test Beta test: customer pilot Parallel testing（ ）: new system operates in parallel with old system,Results of acceptance tests驗收測試的結(jié)果,After acceptance testing, the customer tells us which requirements are not sa

42、tisfied and which must be deleted, revised, or added because of changing needs. Configuration management staff identify these changes and record the consequent modifications to design, implementation, and testing.在驗收測試之后，顧客告訴我們哪些需求還沒有滿足，以及由于需求變化哪些需求必須刪除、修改或增加。配置人員確認(rèn)這些改變，并記錄改動所引起的對設(shè)計、實現(xiàn)和測試的修改。,9.6 in

43、stallation testing 安裝測試,The final round of testing involves installing the system at user sites. Installation test require us to work with the customer to determine what tests are needed on site. The test cases assure the customer that the system is complete and that all necessary files and devices

44、are present. The tests focus on two things: completeness of the installed system and verification of any functional or nonfunctional characteristics that may be affected by site conditions. 最后一輪測試是在用戶工作地點安裝系統(tǒng)。安裝測試需要我們與顧客一起判定需要在現(xiàn)場進行什么測試。測試實例使顧客確信該系統(tǒng)是完備的，所有必需的文件和設(shè)備都已齊備。測試的重點有兩個：所安裝系統(tǒng)的完備性、對任何可能被現(xiàn)場條件影響的

45、功能性或非功能性特性進行驗證。,9.7 Test documentation測試文檔,Test plan（ ）: describes system and plan for exercising all functions and characteristics Test specification and evaluation （ ）: details each test and defines criteria for evaluating each feature Test description（ ）: test data and procedures for each test Te

46、st analysis report（ ）: results of each test,測試中產(chǎn)生的文檔,一個測試計劃的各個部分,測試計劃的參考模板,測試用例,測試報告的參考模板,性能測試用例的參考模板,Problem report forms 問題報告表,Location地點：該問題發(fā)生在何處？ Timing 計時：它于何時發(fā)生？ Symptom癥狀：觀察到什么？ End result最終結(jié)果：后果是什么？ Mechanism機制：它是怎樣發(fā)生的？ Cause原因：它為什么會發(fā)生？ Severity嚴(yán)重級別：用戶或事務(wù)被影響的程度怎樣？ Cost成本：它的代價有多大,9.8 Testing safety-critical systems測試安全攸關(guān)的系統(tǒng),De