Unity Test Tools 1.2 have been released

Assertion Component

Code stripping

We’ve been hearing from lots of people that the main issue that prevents them from using the assertion component is the lack of the option to strip the code out of the release builds. As we faced this issue, we came up with a solution, which is (duh!) very trivial.

We implemented scene post-process mechanism that removes all components from the builded scenes. Nevertheless, the code of the assertion is still being included in the builds. This is due to Unity’s nature and the potency of managed languages (you still might want to use reflection on non-referenced code, don’t you?). But no worries! It’s not anything you can’t deal with.

The solution to completely strip the code from your build would be to implement your own build pipeline and build your project with BuildPipeline.BuildPlayer. See an example of custom build pipeline implementation below:

using UnityEngine;
using UnityEditor;

public class StripCodeAndBuild
{
   private static string assetsPath = "Assets/UnityTestTools";
   private static string buildTempPath = "Assets/Editor/UnityTestTools";

   [MenuItem ("Unity Test Tools/StripCodeAndBuild")]
   public static void StripCodeAndBuildStandalone ()
   {
      AssetDatabase.CreateFolder ("Assets", "Editor");
      var result = AssetDatabase.MoveAsset (assetsPath, buildTempPath);

      if (string.IsNullOrEmpty (result))
      {
         BuildPipeline.BuildPlayer (new[] {EditorApplication.currentScene},
                                    @"C:\temp\release.exe",
                                    BuildTarget.StandaloneWindows,
                                    BuildOptions.None);
         result = AssetDatabase.MoveAsset (buildTempPath, assetsPath);
         if (string.IsNullOrEmpty (result))
            AssetDatabase.Refresh ();
         else
            Debug.LogWarning (result);
      }
      else
      {
         Debug.LogWarning (result);
      }
   }
}

The StripCodeAndBuildStandalone will move the whole UnityTestTools folder to a path underEditor folder, build a WindowsStandalone player, and then move the tools back to it’s original location. Any file under Editor folder is not included in the build.

We hope this will make some people happy and encourage them to use the assertion as a first barrier of defence against bugs!

Performance

As an assertion, the component should have as little performance impact on runtime as possible. We took a closer look and redesigned Assertion Component’s background code to make it run faster… almost 5 times faster! That is at least for a benchmark scene with 1000 check calls per frame.

Assertion Component overhead before optimization (~10ms per 1000 calls):

After optimization  (~2ms per 1000 calls):

Integration tests

Hierarchical structure

If you like nesting and grouping your test, this feature is for you! We made it possible to put one integration tests under another one (just like you would with GameObjects) to clear out the structure of tests. Once a test becomes a parent for another tests, it is treated as a test group. A test group can contain tests or more test groups. See the following screenshot:

 

The tests on the example scene were grouped by their result. Also the result triggers were placed accordingly. We hope this feature will make test scenes with large amount of tests more readable and easier to manage.

Platform Runner

Once you have your integration tests written, a natural thing to do is to run them on your the platform you are developing for. The Platform Runner was created to help you in automating this process. It will build selected scenes for the platform you choose and run it. Additionally, it allows you specify a path to save the results at.

A quick run option for running current scene is also available from the menu.

Reporting Results

Until now, the only way to get the results of a test run was to read the result file that was generated… if it was generated. Not every platform Unity can build for supports file system, and if they do, sometime it get tricky to get use of it (due to system permissions or accessing the file on the device).

To address that issue, we added a result report showed right on the screen of a device you are running the tests on. I will show you a list of failed tests and the logs that appeared while the test was running.

Unit Tests

Background runner

For your convenience we have rewritten the background runner mechanism to run unit tests even though the Unit Test Runner window is not focused. It will run seamlessly and notify you only if any of your unit tests failed.

Undo System

Instantiating GameObjects in a unit test is not a good idea in general, however, sometimes you might have no other choice (or no will to refactor your code!). The Unit Test Runner gives an option to run unit tests on a separate scene, which is a fast and easy way to clean the ‘pollution’  from the scene. This however gives some overhead of reopening your current scene after every run. To make unit tests execution fast and seamless you should avoid running them on a separate scene and use Unity’s undo system to backout any changes to the scene you make in your unit tests. Simply register them with Undo class or utilize the UnityUnitTest base class that comes with the tools.

Continuous Integration Systems accomodation

One of the most commonly asked questions from you are related to continuous integration systems. It’s quite obvious that once you have the automation written, you don’t want to run them manually every time but delegate this task to the machines. It’s called an automation for a reason.

To accommodate some of your requests we redesigned the batch runner for both unit tests and integration tests runners. We added possibility to specify result file path from as a command line argument and enabled integration tests to be run and build in a player from batch. See the documentation for changes.

Summary

Once again big hugs to all of you that contributed, shared their opinion and reported issues. There is a reason we released  the tools via the Asset Store and it’s all about you being involved in development. It allows us to rapidly respond for you feedback and accommodate your needs. We make the tools for you and we encourage you to contribute. Share your opinions and experience of using Unity Test Tools at Unity Forum or via Twitter (#UnityTestTools).

Along with working on the tools, we actively develop best practices and patterns for testing games. Stay tuned for a series of blog posts about that!

And remember: Don’t just make games, make them better!