Lesson 1 - Software Testing Engineering

Software Engineering • Module 5

Software Testing Engineering

Lesson 1 | Prof. Afonso Brandão

Unit Tests

AAA and Parameters

Black Box and E2E

RTM and BDD

What Is Testing?

Testing is evidence that the implemented behavior still matches the expected behavior

Testing is not only about finding bugs.
Testing is about verifying business behavior, risk, and quality.
Each important test should protect something that matters to the project.

          In this course, tests are evidence for the business drivers already modeled by the team.
        

Testing and Modeling

The test strategy should start from business drivers

What Comes From Modeling

Critical flows
Domain rules
Operational risks

What Testing Must Prove

Behavior is correct
Rules are enforced
Failures are handled safely

If a business driver says reliability matters in checkout, then tests must verify checkout behavior under normal and failure conditions.

Unit Tests

Unit tests protect domain rules with fast and precise feedback

Good Targets

Credit limit rules
Discount policies
Permission checks

Why They Matter

Fast to run
Cheap to maintain
Great for business invariants

Unit Tests and Business Drivers

The point is not method coverage, but rule protection

If fraud prevention is a driver, unit test fraud scoring rules.
If pricing accuracy is a driver, unit test discount and tax calculation rules.
If access control is a driver, unit test permission decisions.

A good unit test asks: does this business rule remain true?

AAA Structure

Arrange, Act, Assert keeps a test readable and focused

AAA Steps

Arrange: prepare inputs and state
Act: execute one behavior
Assert: verify one observable result

Why It Helps

Clearer intent
Less noisy tests
Fewer weak assertions

# Arrange
customer = Customer(limit=1000)
order = Order(total=1300)

# Act
result = approve_order(customer, order)

# Assert
assert result.status == "rejected"

AAA Example

Short tests are easier to trust

def test_order_above_credit_limit_is_rejected():
    # Arrange
    customer = Customer(limit=1000)
    order = Order(total=1300)

    # Act
    result = approve_order(customer, order)

    # Assert
    assert result.status == "rejected"
    assert result.reason == "credit_limit"

Parameterized Tests

One rule, many examples

Use Them For

Thresholds
Boundary values
Validation matrices

Main Benefit

Less duplicated setup
Rules become visible as data
Boundary cases stay explicit

@pytest.mark.parametrize(
    "amount,expected",
    [(999, "approved"), (1000, "approved"), (1001, "rejected")]
)
def test_credit_limit_rule(amount, expected):
    ...

Parameterized Test Example

Business rules often look like tables

@pytest.mark.parametrize(
    "amount,expected",
    [(999, "approved"), (1000, "approved"), (1001, "rejected")]
)
def test_credit_limit_rule(amount, expected):
    customer = Customer(limit=1000)
    result = approve_order(customer, Order(total=amount))
    assert result.status == expected

White-Box Testing

Designed with knowledge of internal logic

Useful for branch coverage and defensive paths.
Common in unit tests and low-level component tests.
Main question: did we exercise the internal logic safely?

Black-Box Testing

Designed from behavior, contracts, and outcomes

Useful for APIs, user journeys, and integrations.
Focus on inputs, outputs, state changes, and side effects.
Main question: does the system behave as promised?

Most business drivers are expressed as black-box expectations.

E2E Testing

End-to-end tests should cover only the workflows that must never break

Start from a critical user journey.
Use realistic auth, persistence, and infrastructure.
Assert business outcomes, not only UI clicks.

What Behavior Means in E2E

Behavior is more than a successful response

Verify Outcomes

Request accepted or rejected correctly
State persisted correctly
Expected events emitted

Verify Safety

Errors visible to the user
Logs or audit trail created
System left in a valid state

E2E Example

A single journey can have multiple observable behaviors

Journey: user resets password

Verify:
- token validation
- password update
- session invalidation
- audit log emission
- visible success or failure response

Testcontainers

Use real disposable infrastructure for integration tests

Good Uses

PostgreSQL
Redis
Kafka or RabbitMQ

Main Benefit

Real configuration
Real schema behavior
Real network assumptions

from testcontainers.postgres import PostgresContainer

with PostgresContainer("postgres:16") as postgres:
    db_url = postgres.get_connection_url()
    repo = OrderRepository(db_url)
    assert repo.healthcheck() is True

VCR

Replay real HTTP exchanges without calling the provider every run

Useful for paid or rate-limited APIs.
Useful for slow or unstable third-party services.
Better than inventing a fake interaction from scratch.

import vcr
import requests

with vcr.use_cassette("fixtures/payment.yaml"):
    response = requests.get("https://api.example.com/payment/42")
    assert response.status_code == 200

VCR replays captured real exchanges. It is not the same as building a mock by hand.

Why Not Use Mocks Here?

Mocks are useful in unit tests, but weak for integration confidence

Main Risks

They reproduce our assumptions
They hide contract problems
They hide timeout and serialization issues

Better Choice

Mocks for unit tests
Testcontainers for internal infrastructure
VCR or contract tests for external HTTP

          If the goal is integration confidence, replacing the integration with a mock defeats the purpose.
        

RTM

Requirements Traceability Matrix connects requirement, test, and evidence

Shows which requirement is covered by which test case.
Makes coverage gaps visible.
Links project decisions to quality evidence.

RTM Example

Coverage becomes explicit and reviewable

Business Driver	Requirement	Test Case	Status	Evidence	Commit
Reliability	Reset token expires in 30 minutes	TC-07	Passed	`report.md`	`abc123`
Security	Only the account owner can update the profile	TC-08	Passed	`test-output.md`	`def456`
Operational Traceability	Password reset must generate an audit log	TC-09	Passed	`audit-check.md`	`ghi789`

Behavior Tests and BDD

Behavior-Driven Development turns business expectations into executable examples

Start from business language.
Describe examples stakeholders can understand.
Reduce ambiguity before coding.

Given / When / Then

This structure organizes behavior as context, action, and outcome

How It Works

Given: the initial context
When: the action performed
Then: the expected outcome

Why It Helps

Forces clear scenarios
Reduces ambiguous acceptance criteria
Works well with product and engineering together

Gherkin

Gherkin is the textual format commonly used to write BDD scenarios

Uses keywords such as Feature, Scenario, Given, When, and Then.
Focuses on readable behavior descriptions, not implementation details.
Helps create shared understanding between technical and non-technical stakeholders.

Gherkin is useful when the team wants acceptance criteria to become executable scenarios.

Gherkin Example

A complete feature can express rules, restrictions, and multiple scenarios

Feature: Password reset
  In order to recover account access
  As a registered user
  I want to reset my password safely

  Background:
    Given the password reset service is available
    And the audit log service is available

  Rule: Reset tokens expire after 30 minutes

  Scenario: Valid token allows password update
    Given a valid reset token created 10 minutes ago
    And the user is active
    When the user submits a strong new password
    Then the API returns 200
    And the password is updated
    And all previous sessions are invalidated
    And an audit event is recorded

  Scenario: Expired token is rejected
    Given a valid reset token created 31 minutes ago
    When the user submits a strong new password
    Then the API returns 401
    And the password is not updated
    And the response contains "token expired"

  Scenario Outline: Weak passwords are rejected
    Given a valid reset token created 5 minutes ago
    When the user submits ""
    Then the API returns 422
    And the password is not updated
    And the response contains ""

    Examples:
      | password | reason              |
      | 123456   | password too weak   |
      | abcdef   | password too weak   |
      | short1   | password too short  |

From Gherkin to Executable Tests

A BDD library maps each step sentence to code

Python Option

Install with pip install behave
Write feature files in .feature format
Create step definitions in Python

pip install behave

How Translation Works

Given, When, and Then lines are matched to Python functions
The framework executes those functions in sequence
Assertions inside the step code make the scenario pass or fail

Step Definitions Example

The feature file stays readable while the Python code performs the real actions

from behave import given, when, then
import requests

@given("a valid reset token created 10 minutes ago")
def step_valid_token(context):
    context.token = create_reset_token(age_in_minutes=10)

@when("the user submits a strong new password")
def step_submit_password(context):
    context.response = requests.post(
        f"{context.base_url}/password-reset",
        json={"token": context.token, "password": "StrongPass123!"}
    )

@then("the API returns 200")
def step_status_200(context):
    assert context.response.status_code == 200

@then("the password is updated")
def step_password_updated(context):
    assert password_was_updated_for(context.token)

Gherkin describes the behavior. The step definitions connect those sentences to executable code and assertions.

How to Write a Test Case

A good test case is clear, reproducible, and project-related

Objective
Precondition
Test procedure
Expected result
Obtained result
Post-condition

Test Case Example

The structure should be specific enough that another person can reproduce it

Test case: Reject duplicate signup

Objective:
Prevent duplicate user registration

Precondition:
User email already exists in the database

Procedure:
Send POST /users with the existing email

Expected result:
API returns 409 and no new user is created

Graded Activity

In-class weighted assignment

Describe one project-related test case.
Include objective, precondition, procedure, expected result, obtained result, and post-condition.
Automate the test using a tool of your choice.

Graded Activity Submission

What must be in the GitLab repository

Repository Content

Markdown report
Automated test code
RTM entry

Required Evidence

GitLab link
At least 3 commits
Traceability from requirement to evidence

          The goal is not only to send a request. The goal is to document and prove a real project behavior.
        

Questions?

Next: Operating Systems and Virtualization

Sobre este encontro

Software Testing Engineering · Prof. Afonso

Objetivo de aprendizagem

Ao final do encontro, o estudante deve ser capaz de aplicar ao projeto os conceitos de Software Testing Engineering. Escopo do encontro: Estrategia de testes baseada em risco, evidencias de qualidade, confiabilidade e engenharia de testes de software.

Estratégia do encontro

Exposição dialogada dos conceitos, alternada com aplicação guiada ao projeto do grupo, e fechamento com verificação do entendimento.

Estrutura do encontro

Introduction
Reading Guide
1) What Testing Means
2) Testing and Business Drivers
3) Unit Tests and Domain Rules
4) AAA: Arrange, Act, Assert
5) Parameterized Tests
6) White-Box and Black-Box Perspectives