Skip to main content
AutoRABIT, Inc.

Job Results

This screen allows you to track the execution of your Continous Integration (CI) processes.

Here you can trigger a new cycle and check the latest metadata changes, deployment warnings and check-ins.

Real-time Build and Deployment log can be accessed for both scheduled and forced processes. Build components can be downloaded locally and you can also see Test results, Code coverage reports and Regression trends from Dashboard.

Icon Description
ci-03.png Refresh icon allows you to refresh the status of the current running Build/Deployment without refreshing the entire browser
This Status Icon indicates that the Build/Deployment is Completed Successfully
This Status Icon indicates that the Build/Deployment is still In Progress
This Status Icon indicates that the Build/Deployment is Failed
This Menu Icon contains sub sections like: View Changes, View Dashboard, Deployment Warnings, View Check-ins & Download Build
This Cycle indicates the number of cycle which is currently running or last run
View Log under Deployment allows you to view the real-time log of the Deployment
View Changes allows you to view the changes made to the current Build
View Dashboard provides comprehensive information about the project
Deployment Warnings allows you to view the deployment warnings that occur during the deployment of the builds
View Check-ins allows to get a comprehensive
Download Build allows you to download the Build Package when the Build is success
The drop down here allows to switch between the CI Jobs created.
 
1. View Changes

View Changes allows you to view the changes fetched from the Source based on the CI Job configuration.

2. View Dashboard

AutoRABIT project dashboard provides comprehensive information about the project by fetching information from all the integrated systems. Dashboard works like a complete team system enabling the team to be more agile and effective.

Dashboard provides information such as:

  1. Test cases that have failed or succeeded.
  2. Percentage of requirements that are automated.
  3. Requirements that have failed or the requirements that are functionally working.
  4. Open issues present at a point of time.
  5. The current code coverage.
  6. Performance of a build over a period of time in various aspects.
A. To view the dashboard of a build

In the Job Result homepage, Click View Dashboard. The following screen gets displayed:

On the left hand side, overall project and list of modules associated with the project are displayed.

  • Click Overall Project and it displays information about the complete project.
  • If you have created modules during the creation of the project, click on a module and it displays information about the module.
B. The dashboard has following sections:
  • Test Results
  • Test Category Summary
  • Code coverage
  • ALM Integrations
  • Requirements
  • Functional
  • Defects
  • Regression trends
C. Test Results

Test results report provide a comprehensive view of failed tests, the details about the failure, what tests have succeeded, and related information.

Click View of Test results to get the detailed test results of a build. In the succeeding window you can see the following:

D. Test Reports

Functional test reports are the reports published by functional tests like Selenium and QTP, after a test cycle.

The default view of test report view is a functional test report.

The test results page has extensive filters with ability to view results as per module, category, test type, browser, and so on.

The table on the right pane is a detailed error report with a summary of the selected criteria. The report contains the package name, class name, test case name, test description, browser in which the test case is executed, test category, view details and create (if there is an integration with issue tracking system).

Three types of reports are generated for a build, to view the reports:

  • Click Error to view the error reports.
  • Click Failure to view the failure report.
  • Click Success to view the success reports.

  • To view the complete error details of a particular test case, click View Details of each row. An error report gets displayed as shown below:

  • At any point of time, the user has an option to view the overall summary of the test results by clicking Overall Summary present in the error report.

The overall summary of error report would look as follows:

E. Viewing Test category Summary

AutoRABIT has come up with a convenient way of categorizing the test cases in the context of application sanity, to enable easy feedback cycles.

Test category limits are set during the creation of the project.

The objective of test categorization is to let the team prioritize the test plans so that they can effectively plan their test execution ranging from sanity check to exhaustive testing.

 

F. Code Coverage

The Code Coverage report provides information about the apex tests that are run, the classes that are covered, and the assertions that have failed and provides a percentage of the code that is covered by the test execution.

G. Viewing PMD

AutoRABIT runs the PMD code analysis and provides a report of the violations of the static rule-set and common code flaws like unnecessary object creation, and so on.

H. Viewing Class Coverage Report

It provides overall code coverage of the application as well as coverage at the class-level with details of the unit tests that have failed.
It covers:

  • Class Coverage
  • Test Coverage

1. Class Coverage

Click View in the Code Coverage section at the far right on the Dashboard. The following screen is displayed:

1. TestCoverage

Test Coverage shows the tests have been successful or not.

Click Test Coverage. The following window is displayed

 

J. Viewing ALM Integrations

ALM Integrations section in the dashboard provides information about the test coverage with requirements, functional test cases, and the current status of the defect backlog.

This would help the Project Managers to know about the current status of the overall project and helps in taking the right decisions about the release.

K. Viewing Requirements Coverage

The Requirements section shows the test coverage with requirements defects from the integrated requirements management system.

Icon Description
Indicates that all requirements, mapped when defining the project, have at least one automated test case associated with it.
Indicates that no test case has failed from the test cases mapped.
Indicates the percentage of requirements that are not mapped to automated test cases.

Click View for a detailed report:

Viewing Functional Settings

The FUNCTIONAL section shows the test coverage with functional test cases.

Icon

Description

Indicates that all requirements, mapped when defining the project, have at least one automated test case associated with it.

Indicates that no test case has failed from the test cases mapped.

Indicates the percentage of requirements that are not mapped to automated test cases.

The bar in this section is a combination of all the three, which is the number of automated test cases, number of failed test cases, and the percentage of requirements that are not mapped to test cases.

Click View to see a detailed report shown below:

The icon in the report in status column shows the following.

Icon

Description

Associated test case has failed in test execution

Associated test case has passed in test execution

Functional test case doesn’t have any mapping with automated test case

L. Viewing Defects

The DEFECTS section shows the number of closed defects and open defects from the integrated issue tracking system.

The bar shown in the image below represents the number of open and closed defects in percentage format.

Icon

Description

Total number of open defects

Total number of closed defects

Click View to obtain a detailed report:

M. Interpreting Code Coverage

The Code Coverage report provides information about the apex tests that were run, the classes that were covered, and assertions that failed and provides a percentage of the code that is covered by the test execution.

Click View to obtain the code coverage report:

N. Regression Trends

The regression test trend shows the success percentage of tests against various build cycles. The percentage is plotted as a line graph shown above.
Regression test trends are used as a key indicator to highlight the consistency of the application quality over the last few builds. It is widely used as a key benchmarking tool by the release managers at the time of QA or UAT phases to take decisions with respect to the application release.

3. Deployment Warnings

Deployment Warnings allows you to view the deployment warnings that occur during the deployment of the builds, please refer to below screenshot for a sample.

4. View Check-ins

View Check-ins allows to get a comprehensive Check-in Summary of your CI Job.

The details displayed include:

  1. The header displays the following details:

    1. Project Name - The name of the project.
    2. Build No. – This is the build number stored in the source control (GIT/ SVN).
    3. Build Date & Time – This the date and time when the build was triggered.
    4. Number of files modified – The number of the source files that were changed for that specific build.
    5. Number of check-ins – This is the number of check-ins that were made by the developers for this particular build.
    6. LOC modified – This is the number of lines of code that have been modified during the check-ins for this build.

 

  1. On the left pane Check-in history is displayed, which displays details of all check-ins made for that build:
    1. Revision No.: The revision number from source control for the corresponding check-in.
    2. Committed By – The user id of the user who committed the change.

  1. Select the revision number from the left pane to view the check-in details associated with it. This makes the code and project review a very comprehensive operation rather than having to be done in a standalone mode.

  1. Check-in Comments: The developers usually provide comments during check-in and this is indicative of the change that goes into the source with the check-in.
  2. Modified Files: Displays all the files where changes are made. 
    • Click on the change set icon to open a comparison tool that displays the specific changes in the code.
    • The modifications display the lines added and removed as shown above between successive builds.
  3. Deleted Files: Displays the deleted files. 
  4. Added Files: Displays all the files added during the current check-in.
    • Click the change set icon to view the code that has been added and deleted in the file.


5. Download Build

Download Build allows you to download the Build Package when the Build is success.


    

  • Was this article helpful?