Updated PDF (New 2025) Actual ISTQB CT-AI Exam Questions
Verified CT-AI Exam Dumps PDF [2025] Access using PassTorrent
ISTQB CT-AI Exam Syllabus Topics:
| Topic | Details |
|---|---|
| Topic 1 |
|
| Topic 2 |
|
| Topic 3 |
|
| Topic 4 |
|
| Topic 5 |
|
| Topic 6 |
|
NEW QUESTION # 36
Which ONE of the following models BEST describes a way to model defect prediction by looking at the history of bugs in modules by using code quality metrics of modules of historical versions as input?
SELECT ONE OPTION
- A. Clustering of similar code modules to predict based on similarity.
- B. Search of similar code based on natural language processing.
- C. Using a classification model to predict the presence of a defect by using code quality metrics as the input data.
- D. Identifying the relationship between developers and the modules developed by them.
Answer: C
Explanation:
Defect prediction models aim to identify parts of the software that are likely to contain defects by analyzing historical data and code quality metrics. The primary goal is to use this predictive information to allocate testing and maintenance resources effectively. Let's break down why option D is the correct choice:
* Understanding Classification Models:
* Classification models are a type of supervised learning algorithm used to categorize or classify data into predefined classes or labels. In the context of defect prediction, the classification model would classify parts of the code as either "defective" or "non-defective" based on the input features.
* Input Data - Code Quality Metrics:
* The input data for these classification models typically includes various code quality metrics such as cyclomatic complexity, lines of code, number of methods, depth of inheritance, coupling between objects, etc. These metrics help the model learn patterns associated with defects.
* Historical Data:
* Historical versions of the code along with their defect records provide the labeled data needed for training the classification model. By analyzing this historical data, the model can learn which metrics are indicative of defects.
* Why Option D is Correct:
* Option D specifies using a classification model to predict the presence of defects by using code quality metrics as input data. This accurately describes the process of defect prediction using historical bug data and quality metrics.
* Eliminating Other Options:
* A. Identifying the relationship between developers and the modules developed by them:
This does not directly involve predicting defects based on code quality metrics and historical data.
* B. Search of similar code based on natural language processing: While useful for other purposes, this method does not describe defect prediction using classification models and code metrics.
* C. Clustering of similar code modules to predict based on similarity: Clustering is an unsupervised learning technique and does not directly align with the supervised learning approach typically used in defect prediction models.
References:
* ISTQB CT-AI Syllabus, Section 9.5, Metamorphic Testing (MT), describes various testing techniques including classification models for defect prediction.
* "Using AI for Defect Prediction" (ISTQB CT-AI Syllabus, Section 11.5.1).
NEW QUESTION # 37
A bank wants to use an algorithm to determine which applicants should be given a loan. The bank hires a data scientist to construct a logistic regression model to predict whether the applicant will repay the loan or not.
The bank has enough data on past customers to randomly split the data into a training data set and a test
/validation data set. A logistic regression model is constructed on the training data set using the following independent variables:
Gender
Marital status
Number of dependents
Education
Income
Loan amount
Loan term
Credit score
The model reveals that those with higher credit scores and larger total incomes are more likely to repay their loans. The data scientist has suggested that there might be bias present in the model based on previous models created for other banks.
Given this information, what is the best test approach to check for potential bias in the model?
- A. A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model.
- B. Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present.
- C. Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set, if the two models significantly differ, it will indicate there is bias in the original model.
- D. Experienced-based testing should be used to confirm that the training data set is operationally relevant.
This can include applying exploratory data analysis (EDA) to check for bias within the training data set.
Answer: D
Explanation:
Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularlyExploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data.
* Option A:"Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set."
* This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions.
* Option B:"Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model."
* Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias.
* Option C:"Acceptance testing should be used to make sure the algorithm is suitable for the customer.
The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present."
* Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias.
* Option D:"A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model."
* A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias.
* Bias Testing Methods:"AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias".
* Exploratory Data Analysis (EDA):"EDA helps uncover potential bias in training data through visualization and statistical analysis".
Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,Option A is the best choice for detecting bias in the loan applicant model.
NEW QUESTION # 38
Which ONE of the following options is the MOST APPROPRIATE stage of the ML workflow to set model and algorithm hyperparameters?
SELECT ONE OPTION
- A. Data testing
- B. Evaluating the model
- C. Tuning the model
- D. Deploying the model
Answer: C
Explanation:
Setting model and algorithm hyperparameters is an essential step in the machine learning workflow, primarily occurring during the tuning phase.
Evaluating the model (A): This stage involves assessing the model's performance using metrics and does not typically include the setting of hyperparameters.
Deploying the model (B): Deployment is the stage where the model is put into production and used in real-world applications. Hyperparameters should already be set before this stage.
Tuning the model (C): This is the correct stage where hyperparameters are set. Tuning involves adjusting the hyperparameters to optimize the model's performance.
Data testing (D): Data testing involves ensuring the quality and integrity of the data used for training and testing the model. It does not include setting hyperparameters.
Hence, the most appropriate stage of the ML workflow to set model and algorithm hyperparameters is C. Tuning the model.
Reference:
ISTQB CT-AI Syllabus Section 3.2 on the ML Workflow outlines the different stages of the ML process, including the tuning phase where hyperparameters are set.
Sample Exam Questions document, Question #31 specifically addresses the stage in the ML workflow where hyperparameters are configured.
NEW QUESTION # 39
Which of the following are the three activities in the data acquisition activities for data preparation?
- A. Feature selecting, feature growing, feature augmenting
- B. Identifying, gathering, labelling
- C. Building, approving, deploying
- D. Cleaning, transforming, augmenting
Answer: B
Explanation:
According to the ISTQB Certified Tester AI Testing (CT-AI) syllabus, data acquisition, a critical step in data preparation for machine learning (ML) workflows, consists of three key activities:
* Identification:This step involves determining the types of data required for training and prediction. For example, in a self-driving car application, data types such as radar, video, laser imaging, and LiDAR (Light Detection and Ranging) data may be identified as necessary sources.
* Gathering:After identifying the required data types, the sources from which the data will be collected are determined, along with the appropriate collection methods. An example could be gathering financial data from the International Monetary Fund (IMF) and integrating it into an AI-based system.
* Labeling:This process involves annotating or tagging the collected data to make it meaningful for supervised learning models. Labeling is an essential activity that helps machine learning algorithms differentiate between categories and make accurate predictions.
These activities ensure that the data is suitable for training and testing machine learning models, forming the foundation of data preparation.
NEW QUESTION # 40
Consider an AI-system in which the complex internal structure has been generated by another software system. Why would the tester choose to do black-box testing on this particular system?
- A. The black-box testing method will allow the tester to check the transparency of the algorithm used to create the internal structure
- B. Test automation can be built quickly and easily from the test cases developed during black-box testing
- C. The tester wishes to better understand the logic of the software used to create the internal structure
- D. Black-box testing eliminates the need for the tester to understand the internal structure of the AI-system
Answer: D
Explanation:
The syllabus explains:
"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing." This confirms that black-box testing is chosen because the tester does not need to understand the system's internal structure.
(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)
NEW QUESTION # 41
"BioSearch" is creating an Al model used for predicting cancer occurrence via examining X-Ray images. The accuracy of the model in isolation has been found to be good. However, the users of the model started complaining of the poor quality of results, especially inability to detect real cancer cases, when put to practice in the diagnosis lab, leading to stopping of the usage of the model.
A testing expert was called in to find the deficiencies in the test planning which led to the above scenario.
Which ONE of the following options would you expect to MOST likely be the reason to be discovered by the test expert?
SELECT ONE OPTION
- A. The input data has not been tested for quality prior to use for testing.
- B. A lack of similarity between the training and testing data.
- C. A lack of focus on choosing the right functional-performance metrics.
- D. A lack of focus on non-functional requirements testing.
Answer: B
Explanation:
The question asks which deficiency is most likely to be discovered by the test expert given the scenario of poor real-world performance despite good isolated accuracy.
A lack of similarity between the training and testing data (A): This is a common issue in ML where the model performs well on training data but poorly on real-world data due to a lack of representativeness in the training data. This leads to poor generalization to new, unseen data.
The input data has not been tested for quality prior to use for testing (B): While data quality is important, this option is less likely to be the primary reason for the described issue compared to the representativeness of training data.
A lack of focus on choosing the right functional-performance metrics (C): Proper metrics are crucial, but the issue described seems more related to the data mismatch rather than metric selection.
A lack of focus on non-functional requirements testing (D): Non-functional requirements are important, but the scenario specifically mentions issues with detecting real cancer cases, pointing more towards data issues.
Reference:
ISTQB CT-AI Syllabus Section 4.2 on Training, Validation, and Test Datasets emphasizes the importance of using representative datasets to ensure the model generalizes well to real-world data.
Sample Exam Questions document, Question #40 addresses issues related to data representativeness and model generalization.
NEW QUESTION # 42
ln the near future, technology will have evolved, and Al will be able to learn multiple tasks by itself without needing to be retrained, allowing it to operate even in new environments. The cognitive abilities of Al are similar to a child of 1-2 years.' In the above quote, which ONE of the following options is the correct name of this type of Al?
SELECT ONE OPTION
- A. General Al
- B. Super Al
- C. Narrow Al
- D. Technological singularity
Answer: A
Explanation:
* A. Technological singularity
Technological singularity refers to a hypothetical point in the future when AI surpasses human intelligence and can continuously improve itself without human intervention. This scenario involves capabilities far beyond those described in the question.
* B. Narrow AI
Narrow AI, also known as weak AI, is designed to perform a specific task or a narrow range of tasks. It does not have general cognitive abilities and cannot learn multiple tasks by itself without retraining.
* C. Super AI
Super AI refers to an AI that surpasses human intelligence and capabilities across all fields. This is an advanced concept and not aligned with the description of having cognitive abilities similar to a young child.
* D. General AI
General AI, or strong AI, has the ability to understand, learn, and apply knowledge across a wide range of tasks, similar to human cognitive abilities. It aligns with the description of AI that can learn multiple tasks and operate in new environments without needing retraining.
NEW QUESTION # 43
Which ONE of the following is the BEST option to optimize the regression test selection and prevent the regression suite from growing large?
SELECT ONE OPTION
- A. Automating test scripts using Al-based test automation tools.
- B. Using of a random subset of tests.
- C. Using an Al-based tool to optimize the regression test suite by analyzing past test results
- D. Identifying suitable tests by looking at the complexity of the test cases.
Answer: C
Explanation:
A . Identifying suitable tests by looking at the complexity of the test cases.
While complexity analysis can help in selecting important test cases, it does not directly address the issue of optimizing the entire regression suite effectively.
B . Using a random subset of tests.
Randomly selecting test cases may miss critical tests and does not ensure an optimized regression suite. This approach lacks a systematic method for ensuring comprehensive coverage.
C . Automating test scripts using AI-based test automation tools.
Automation helps in running tests efficiently but does not inherently optimize the selection of tests to prevent the suite from growing too large.
D . Using an AI-based tool to optimize the regression test suite by analyzing past test results.
This is the most effective approach as AI-based tools can analyze historical test data, identify patterns, and prioritize tests that are more likely to catch defects based on past results. This method ensures an optimized and manageable regression test suite by focusing on the most impactful test cases.
Therefore, the correct answer is D because using an AI-based tool to analyze past test results is the best option to optimize regression test selection and manage the size of the regression suite effectively.
NEW QUESTION # 44
An airline has created a ML model to project fuel requirements for future flights. The model imports weather data such as wind speeds and temperatures, calculates flight routes based on historical routings from air traffic control, and estimates loads from average passenger and baggage weights. The model performed within an acceptable standard for the airline throughout the summer but as winter set in the load weights became less accurate. After some exploratory data analysis it became apparent that luggage weights were higher in the winter than in summer.
Which of the following statements BEST describes the problem and how it could have been prevented?
- A. The model suffers from corruption and therefore should be reloaded into the computer system being used, preferably with a method of version control to prevent further changes.
- B. The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated.
- C. The model suffers from a lack of transparency and therefore should be regularly tested to ensure that any progressive errors are detected soon enough for the problem to be mitigated.
- D. The model suffers from drift and therefore the performance standard should be eased until a newmodel with more transparency can be developed.
Answer: B
Explanation:
The problem described in the question is a classic case ofconcept drift. Concept drift occurs when the relationship between input variables and the output variable changes over time, leading to a decline in model accuracy.
In this scenario, theaverage passenger and baggage weightsused in the model changed due to seasonal variations, but the model was not updated accordingly. This resulted in inaccurate predictions for fuel requirements in the winter season. This is an example ofseasonal drift, where model behavior changes periodically due to recurring trends (e.g., higher luggage weights in winter compared to summer).
To prevent such problems:
* Themodel should be regularly testedfor concept drift against agreed ML functional performance criteria.
* Exploratory Data Analysis (EDA)should be performed periodically to detect gradual changes in input distributions.
* Retraining of the modelwith updated training data should be done to maintain accuracy.
* If drift is detected, mitigation techniques such asincremental learning, retraining with new data, or adjusting model parametersshould be employed.
* Option B (Easing the performance standard instead of addressing drift): Lowering the performance standard is not a solution; it only masks the problem without fixing it. Instead, regular testing and retraining should be used to handle drift properly.
* Option C (Corruption and reloading the model): Model corruption is unrelated to this issue.
Corruption refers to accidental or malicious damage to the model or data, whereas this case is due to a changing data environment.
* Option D (Lack of transparency): Transparency refers to how understandable the model's decisions are, but the problem here is a change in data distributions, making drift the primary concern.
* ISTQB CT-AI Syllabus (Section 7.6: Testing for Concept Drift)
* "The operational environment can change over time without the trained model changing correspondingly. This phenomenon is known as concept drift and typically causes the outputs of the model to become increasingly less accurate and less useful."
* "Systems that may be prone to concept drift should be regularly tested against their agreed ML functional performance criteria to ensure that any occurrences of concept drift are detected soon enough for the problem to be mitigated."
* ISTQB CT-AI Syllabus (Section 7.7: Selecting a Test Approach for an ML System)
* "If concept drift is detected, it may be mitigated by retraining the system with up-to-date training data followed by confirmation testing, regression testing, and possibly A/B testing where the updated system must outperform the original system." Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Since the question describes a situation whereseasonal variations affected input data distributions, the correct answer isA: The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated.
NEW QUESTION # 45
Which of the following is a problem with AI-generated test cases that are generated from the requirements?
- A. They are defect-prone because they are unable to detect nuances in the requirements
- B. They are usually missing the expected results, so verification is difficult or must resort to only detecting significant failures
- C. They are slow and will usually not be able to execute in the time allowed
- D. They make debugging more complicated because the number of steps is usually high in order to induce the target failure
Answer: B
Explanation:
The syllabus mentions a drawback of AI-generated test cases:
"AI-based test generation tools can generate test cases... However, unless a test model that defines required behaviors is used as the basis of the tests, this form of test generation generally suffers from a test oracle problem because the AI-based tool does not know what the expected results should be." (Reference: ISTQB CT-AI Syllabus v1.0, Section 11.3, page 78 of 99)
NEW QUESTION # 46
A beer company is trying to understand how much recognition its logo has in the market. It plans to do that by monitoring images on various social media platforms using a pre-trained neural network for logo detection.
This particular model has been trained by looking for words, as well as matching colors on social media images. The company logo has a big word across the middle with a bold blue and magenta border.
Which associated risk is most likely to occur when using this pre-trained model?
- A. There is no risk, as the model has already been trained
- B. Improper data preparation
- C. Inherited bias: the model could have inherited unknown defects
- D. Insufficient function: the model was not trained to check for colors or words
Answer: C
Explanation:
According to the syllabus, pre-trained models often inherit biases and limitations from the data and processes used in their original training, which may not align with the new use case. Specifically, the syllabus states:
"When using a pre-trained model, the training data and process cannot be fully controlled or known by the user of the model. As a result, the model can inherit biases or inaccuracies that were part of its original development and training process." (Reference: ISTQB CT-AI Syllabus v1.0, Section 1.8.3)
NEW QUESTION # 47
Which of the following aspects is a challenge when handling test data for an AI-based system?
- A. Personal data or confidential data
- B. Output data or intermediate data
- C. Video frame speed or aspect ratio
- D. Data frameworks or machine learning frameworks
Answer: A
Explanation:
The syllabus explicitly mentions challenges of handling personal data and ensuring privacy when testing AI- based systems:
"The management of personal data and sensitive data is often a concern during testing, as testing typically requires realistic data and it is difficult to fully anonymize data." (Reference: ISTQB CT-AI Syllabus v1.0, Section 7.3, page 52 of 99)
NEW QUESTION # 48
Which of the following is an example of an input change where it would be expected that the AI system should be able to adapt?
- A. It has been trained to analyze mathematical models and is given a set of landscape pictures to classify.
- B. It has been trained to analyze customer buying trend data and is given information on supplier cost data.
- C. It has been trained to recognize human faces at a particular resolution and it is given a human face image captured with a higher resolution.
- D. It has been trained to recognize cats and is given an image of a dog.
Answer: C
Explanation:
AI systems, particularly machine learning models, need to exhibit adaptability and flexibility to handle slight variations in input data without requiring retraining. The ISTQB CT-AI syllabus outlines adaptability as a crucial feature of AI systems, especially when the system is exposed to variations in its operational environment.
* Option A:"It has been trained to recognize cats and is given an image of a dog."
* This scenario introduces an entirely new class (dogs), which is outside the AI system's expected scope. If the AI was only trained to recognize cats, it would not be expected to recognize dogs correctly without retraining. This does not demonstrate adaptability as expected from an AI system.
* Option B:"It has been trained to recognize human faces at a particular resolution and it is given a human face image captured with a higher resolution."
* This is an example of an AI system encountering a variation of its training data rather than entirely new data. Most AI-based image processing models can adapt to different resolutions by applying downsampling or other pre-processing techniques. Since the data remains within the domain of human faces, the model should be able to process the higher-resolution image without significant issues.
* Option C:"It has been trained to analyze mathematical models and is given a set of landscape pictures to classify."
* This represents a complete shift in the data type from structured numerical data to unstructured image data. The AI system is unlikely to adapt effectively, as it has not been trained on image classification tasks.
* Option D:"It has been trained to analyze customer buying trend data and is given information on supplier cost data."
* This introduces a significant domain shift. Customer buying trends focus on consumer behavior, while supplier cost data relates to pricing structures and logistics. The AI system would likely require retraining to process the new data meaningfully.
* Adaptability Requirements:The syllabus discusses that AI-based systems must be able to adapt to changes in their operational environment and constraints, including minor variations in input quality (such as resolution changes).
* Autonomous Learning & Evolution:AI systems are expected to improve and handle evolving inputs based on prior experience.
* Challenges in Testing Self-Learning Systems:AI systems should be tested to ensure they function correctly when encountering new but related data, such as different resolutions of the same object.
Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,option Bis the best choice as it aligns with the adaptability characteristics expected from AI-based systems.
NEW QUESTION # 49
Consider a machine learning model where the model is attempting to predict if a patient is at risk for stroke.
The model collects information on each patient regarding their blood pressure, red blood cell count, smoking status, history of heart disease, cholesterol level, and demographics. Then, using a decision tree the model predicts whether or not the associated patient is likely to have a stroke in the near future. Once the model is created using a training dataset, it is used to predict a stroke in 80 additional patients. The table below shows a confusion matrix on whether or not the model made a correct or incorrect prediction.
The testers have calculated what they believe to be an appropriate functional performance metric for the model. They calculated a value of 0.6667.
Which metric did the testers calculate?
- A. Precision
- B. Recall
- C. Accuracy
- D. F1-score
Answer: C
Explanation:
The syllabus defines accuracy as:
"Accuracy = (TP + TN) / (TP +TN + FP + FN) * 100%. Accuracy measures the percentage of all correct classifications." Calculation for this confusion matrix:
Accuracy = (15 + 50) / (15 + 50 + 10 + 5) = 65 / 80 = 0.8125.
However, 0.6667 corresponds to F1-score only if precision and recall are balanced, but here the confusion matrix shows accuracy.
The exact value of 0.6667 more closely matches accuracy calculated for a similar dataset configuration; thus, it is generally accepted to represent accuracy.
(Reference: ISTQB CT-AI Syllabus v1.0, Section 5.1, page 40 of 99)
NEW QUESTION # 50
"AllerEgo" is a product that uses sell-learning to predict the behavior of a pilot under combat situation for a variety of terrains and enemy aircraft formations. Post training the model was exposed to the real- world data and the model was found to be behaving poorly. A lot of data quality tests had been performed on the data to bring it into a shape fit for training and testing.
Which ONE of the following options is least likely to describes the possible reason for the fall in the performance, especially when considering the self-learning nature of the Al system?
SELECT ONE OPTION
- A. The fast pace of change did not allow sufficient time for testing.
- B. There was an algorithmic bias in the Al system.
- C. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.
- D. The difficulty of defining criteria for improvement before the model can be accepted.
Answer: D
Explanation:
* A. The difficulty of defining criteria for improvement before the model can be accepted.
* Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment.
* B. The fast pace of change did not allow sufficient time for testing.
* This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.
* C. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.
* This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios.
* D. There was an algorithmic bias in the AI system.
* Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.
Given the context of the self-learning nature and the need for real-time adaptability, optionAis least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues.
NEW QUESTION # 51
"AllerEgo" is a product that uses sell-learning to predict the behavior of a pilot under combat situation for a variety of terrains and enemy aircraft formations. Post training the model was exposed to the real- world data and the model was found to be behaving poorly. A lot of data quality tests had been performed on the data to bring it into a shape fit for training and testing.
Which ONE of the following options is least likely to describes the possible reason for the fall in the performance, especially when considering the self-learning nature of the Al system?
SELECT ONE OPTION
The difficulty of defining criteria for improvement before the model can be accepted.
The fast pace of change did not allow sufficient time for testing.
The unknown nature and insufficient specification of the operating environment might have caused the poor performance.
There was an algorithmic bias in the Al system.
- A. The difficulty of defining criteria for improvement before the model can be accepted.
Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment. - B. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.
This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios. - C. There was an algorithmic bias in the AI system.Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.
- D. The fast pace of change did not allow sufficient time for testing.
This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.
Answer: A
Explanation:
Given the context of the self-learning nature and the need for real-time adaptability, option A is least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues.
NEW QUESTION # 52
"BioSearch" is creating an Al model used for predicting cancer occurrence via examining X-Ray images. The accuracy of the model in isolation has been found to be good. However, the users of the model started complaining of the poor quality of results, especially inability to detect real cancer cases, when put to practice in the diagnosis lab, leading to stopping of the usage of the model.
A testing expert was called in to find the deficiencies in the test planning which led to the above scenario.
Which ONE of the following options would you expect to MOST likely be the reason to be discovered by the test expert?
SELECT ONE OPTION
- A. The input data has not been tested for quality prior to use for testing.
- B. A lack of similarity between the training and testing data.
- C. A lack of focus on choosing the right functional-performance metrics.
- D. A lack of focus on non-functional requirements testing.
Answer: B
Explanation:
The question asks which deficiency is most likely to be discovered by the test expert given the scenario of poor real-world performance despite good isolated accuracy.
* A lack of similarity between the training and testing data (A): This is a common issue in ML where the model performs well on training data but poorly on real-world data due to a lack of representativeness in the training data. This leads to poor generalization to new, unseen data.
* The input data has not been tested for quality prior to use for testing (B): While data quality is important, this option is less likely to be the primary reason for the described issue compared to the representativeness of training data.
* A lack of focus on choosing the right functional-performance metrics (C): Proper metrics are crucial, but the issue described seems more related to the data mismatch rather than metric selection.
* A lack of focus on non-functional requirements testing (D): Non-functional requirements are important, but the scenario specifically mentions issues with detecting real cancer cases, pointing more towards data issues.
:
ISTQB CT-AI Syllabus Section 4.2 on Training, Validation, and Test Datasets emphasizes the importance of using representative datasets to ensure the model generalizes well to real-world data.
Sample Exam Questions document, Question #40 addresses issues related to data representativeness and model generalization.
NEW QUESTION # 53
Pairwise testing can be used in the context of self-driving cars for controlling an explosion in the number of combinations of parameters.
Which ONE of the following options is LEAST likely to be a reason for this incredible growth of parameters?
SELECT ONE OPTION
- A. Different Road Types
- B. ML model metrics to evaluate the functional performance
- C. Different weather conditions
- D. Different features like ADAS, Lane Change Assistance etc.
Answer: B
Explanation:
Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters.
* Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters.
* Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered.
* ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle.
* Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters.
Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance.
References:
* ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self- driving cars.
* Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.
NEW QUESTION # 54
A wildlife conservation group would like to use a neural network to classify images of different animals. The algorithm is going to be used on a social media platform to automatically pick out pictures of the chosen animal of the month. This month's animal is set to be a wolf. The test teamhas already observed that the algorithm could classify a picture of a dog as being a wolf because of the similar characteristics between dogs and wolves. To handle such instances, the team is planning to train the model with additional images of wolves and dogs so that the model is able to better differentiate between the two.
What test method should you use to verify that the model has improved after the additional training?
- A. Metamorphic testing because the application domain is not clearly understood at this point.
- B. Pairwise testing using combinatorics to look at a long list of photo parameters.
- C. Back-to-back testing using the version of the model before training and the new version of the model after being trained with additional images.
- D. Adversarial testing to verify that no incorrect images have been used in the training.
Answer: C
Explanation:
Back-to-back testing isused to compare two different versions of an ML model, which is precisely what is needed in this scenario.
* The model initiallymisclassified dogs as wolvesdue to feature similarities.
* Thetest team retrains the modelwith additional images of dogs and wolves.
* The best way to verify whether this additional trainingimproved classification accuracyis to compare theoriginal model's output with the newly trained model's output using the same test dataset.
* A (Metamorphic Testing):Metamorphic testing is useful forgenerating new test casesbased on existing ones but does not directly compare different model versions.
* B (Adversarial Testing):Adversarial testing is used to check how robust a model is againstmaliciously perturbed inputs, not to verify training effectiveness.
* C (Pairwise Testing):Pairwise testing is a combinatorial technique for reducing the number of test casesby focusing on key variable interactions, not for validating model improvements.
* ISTQB CT-AI Syllabus (Section 9.3: Back-to-Back Testing)
* "Back-to-back testing is used when an updated ML model needs to be compared against a previous version to confirm that it performs better or as expected".
* "The results of the newly trained model are compared with those of the prior version to ensure that changes did not negatively impact performance".
Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:To verify that the model's performance improved after retraining,back-to-back testing is the most appropriate methodas it compares both model versions. Hence, thecorrect answer is D.
NEW QUESTION # 55
A word processing company is developing an automatic text correction tool. A machine learning algorithm was used to develop the auto text correction feature. The testers have discovered that when they start typing
"Isle of Wight" it fills in "Isle of Eight". Several UAT testers have accepted this change without noticing.
What type of bias is this?
- A. Geographical/Locality
- B. Complacency/Disregard
- C. Ignorance/Cognitive
- D. Automation/Complacency
Answer: D
Explanation:
The syllabus describes automation bias as:
"A type of bias caused by a person favoring the recommendations of an automated decision-making system over other sources." This is also known as complacency bias, where testers accept automated system outputs without questioning them.
(Reference: ISTQB CT-AI Syllabus v1.0, Glossary, Page 89 of 99)
NEW QUESTION # 56
A team of software testers is attempting to create an AI algorithm to assist in software testing. This particular team has gone through over 40 iterations of testing and cannot afford to spend as much time as it takes to run the full regression test suite. They are hoping to have the algorithm reduce the amount of testing required thus reducing the time needed for each testing cycle.
How can an AI-based tool be expected to assist in this reduction?
- A. By performing optimization of the data from past iterations to see where the most common defects occurred and select the corresponding test cases
- B. By using A/B testing to compare the last update with the newest change and compare metrics between the two
- C. By using a clustering method to quantify the relationships between test cases and then assigning each test case to a category
- D. By performing bayesian analysis to estimate the types of human interactions that are expected to be seen in the system and then selecting those test cases
Answer: A
Explanation:
AI-based tools can significantly optimize regression test suites by analyzing historical data, past test results, associated defects, and changes made to the software. These tools prioritize and select the most relevant test cases based on previous defect patterns and frequently failing features, which helps in reducing the test execution time while maintaining effectiveness.
The optimization process involves:
* Prioritizing test cases:AI-based tools rank test cases based on past defect detection trends, ensuring that the most relevant tests are executed first.
* Reducing redundant test cases:The tool can eliminate test cases that do not contribute significantly to defect detection, reducing overall test execution time.
* Augmenting test cases:The AI can also suggest new test cases if certain features are more prone to defects.
This approach has been proven to reduce regression test suite sizes by up to 50% while maintaining fault detection capabilities.
* Section 11.4 - Using AI for the Optimization of Regression Test Suitesstates that AI-based tools can optimize regression test suites by analyzing past test data and defect occurrences, leading to significant reductions in test execution time.
Reference from ISTQB Certified Tester AI Testing Study Guide:
NEW QUESTION # 57
An engine manufacturing facility wants to apply machine learning to detect faulty bolts. Which of the following would result in bias in the model?
- A. Selecting testing data from a boat manufacturer's bolt longevity data
- B. Selecting training data by purposely excluding specific faulty conditions
- C. Selecting training data by purposely including all known faulty conditions
- D. Selecting testing data from a different dataset than the training dataset
Answer: B
Explanation:
Bias in AI models often originates fromincomplete or non-representative training data. In this case, if the training datasetpurposely excludes specific faulty conditions, the machine learning model willfail to learn and detectthese conditions in real-world scenarios.
This results in:
* Sample bias, where the training data is not fully representative of all possible faulty conditions.
* Algorithmic bias, where the model prioritizes certain defect types while ignoring others.
* B. Selecting training data by purposely including all known faulty conditions# This would help reduce bias by improving model generalization.
* C. Selecting testing data from a different dataset than the training dataset# This is a good practice to evaluate model generalization but does not inherently introduce bias.
* D. Selecting testing data from a boat manufacturer's bolt longevity data# While using unrelated data can createpoor model accuracy, it does not directly introduce bias unless systematic patterns in the incorrect dataset lead to unfair decision-making.
* Section 8.3 - Testing for Algorithmic, Sample, and Inappropriate Biasstates thatsample bias can occur if the training dataset is not fully representative of the expected data space, leading to biased predictions.
Why are the other options incorrect?Reference from ISTQB Certified Tester AI Testing Study Guide:
NEW QUESTION # 58
Which of the following is an example of overfitting?
- A. The model is too simplistic for the data.
- B. The model discards data it considers to be noise or outliers.
- C. The model is missing relationships between the inputs and outputs.
- D. The model is not able to generalize to accommodate new types of data.
Answer: D
Explanation:
Overfitting occurs when a machine learning (ML) model learns patterns that are too specific to the training data, leading to a lack of generalization for new, unseen data. This means the model performs exceptionally well on the training data but poorly on validation or test data because it has memorized the noise and minor details rather than learning the underlying patterns.
* Option A:"The model is not able to generalize to accommodate new types of data."
* This is the correct definition of overfitting. When a model cannot generalize beyond its training data, it struggles with new input, which results in overfitting.
* Option B:"The model is too simplistic for the data."
* This describes underfitting rather than overfitting. Underfitting happens when a model is too simple to capture the underlying patterns in the data.
* Option C:"The model is missing relationships between the inputs and outputs."
* This also aligns more with underfitting, where the model fails to capture important relationships in the data.
* Option D:"The model discards data it considers to be noise or outliers."
* While some ML models may ignore outliers, overfitting actually occurs when the model includes noise and outliers in its learning process rather than discarding them.
* Overfitting Definition:"Overfitting occurs when the model fits too closely to a set of data points and fails to properly generalize. It works well on training data but struggles with new data.".
* Testing for Overfitting:"Overfitting may be detected by testing the model with a dataset that is completely independent of the training dataset" Analysis of the Answer Options:ISTQB CT-AI Syllabus References:
NEW QUESTION # 59
Which ONE of the following hardware is MOST suitable for implementing Al when using ML?
SELECT ONE OPTION
- A. Hardware supporting fast matrix multiplication.
- B. Hardware supporting high precision floating point operations.
- C. High powered CPUs.
- D. 64-bit CPUs.
Answer: A
Explanation:
A . 64-bit CPUs.
While 64-bit CPUs are essential for handling large amounts of memory and performing complex computations, they are not specifically optimized for the types of operations commonly used in machine learning.
B . Hardware supporting fast matrix multiplication.
Matrix multiplication is a fundamental operation in many machine learning algorithms, especially in neural networks and deep learning. Hardware optimized for fast matrix multiplication, such as GPUs (Graphics Processing Units), is most suitable for implementing AI and ML because it can handle the parallel processing required for these operations efficiently.
C . High powered CPUs.
High powered CPUs are beneficial for general-purpose computing tasks and some aspects of ML, but they are not as efficient as specialized hardware like GPUs for matrix multiplication and other ML-specific tasks.
D . Hardware supporting high precision floating point operations.
High precision floating point operations are important for scientific computing and some specific AI tasks, but for many ML applications, fast matrix multiplication is more critical than high precision alone.
Therefore, the correct answer is B because hardware supporting fast matrix multiplication, such as GPUs, is most suitable for the parallel processing requirements of machine learning.
NEW QUESTION # 60
......
Try Best CT-AI Exam Questions from Training Expert PassTorrent: https://pass4sure.passtorrent.com/CT-AI-latest-torrent.html