Skip to main content

Testing your App

Testing practices​

The Overwolf team will review your app when you submit it both for the first time and when you update it. This article we will share how Overwolf's testing process works, what is focused on, and what you should focus on when preparing your app for submission. If you have any questions, reach out via Slack or email: developers@overwolf.com.

app manifest​

Consistent UID​

Make sure that the author and app name are identical in every version. Changes in these parameters will change the UID of your app and cause issues.

Validate your manifest.json​

Make sure to validate your manifest.json. Make sure that your manifest.json file is a strictly valid JSON, without trailing commas, etc. Make sure your file is correct and complete by validating it against Overwolf’s's schema file.

Use one of the following methods:

Validate the schema​

Ensure that your manifest.json is consistent with our schema:

  1. Copy the contents of the app’s manifest.json and paste it under "Input JSON:" in jsonschemavalidator.net.
  2. Copy the contents of our schema and paste it under "Select schema:".
  3. Make sure you have no errors and that you see a green checkmark with the text "No errors found. JSON validates against the schema".

Minimum Overwolf supported version​

Check your app's minimum Overwolf version and make sure that it supports your app's required features. Update the version when new APIs or bug fixes may be required or needed for your app. Verify that this minimum required version is fully rolled out to all users before releasing!

Verify correct Advertisement flags​

  1. "block_top_window_navigation":true—opens the link from the ad in a new window when when the app user clicks on it.
  2. "popup_blocker":true—use this to prevent ads that try to create a popup without the user’s consent.
  3. "mute":true—mute sounds in window.

Verify the 'launcher_icon' property​

  1. Make sure the key:value pair "launcher_icon": "icon.ico" appears in the meta object. Replace icon.ico with the icon file name of the icon in your OPK.
  2. Verify that you have an icon file in your app folder/opk and that its size is less than 150KB.
  3. Make sure that you have the following resolutions for it: 16x16, 32x32,48x48, 256x256.

Verify the 'icon' and 'icon_gray' property​

  1. Make sure the key:value pair "icon": "IconMouseOver.png" and "icon_gray": "IconMouseNormal.png" appear in the meta object. Replace "IconMouseOver.png" and "IconMouseNormal.png" with the names of the files in your app's OPK.
  2. Verify that the files exist in your app folder or opk and the sizes are less than 30KB.
  3. Make sure that each of the files are 256x256 in resolution size.

Verify game_targeting flag existence​

Verify correct launch_events​

  1. Make sure that your app only launches with the games it interacts with.
  2. If your app is set to LaunchWithOverwolf, make sure:
    1. It does not open any visible windows unless a game was opened.
    2. The user explicitly opts-in and can opt-out at any time.

Windows​

Resolution compatibility​

  1. Before you test your app, set the DPI to 100% and window resolution to 1920 x 1080.

  2. Change your screen resolution to the following settings and make sure that the app window stays within the borders of your screen:

    • 1366 x 720 – 100 DPI
    • 1366 x 768 – 100 DPI
    • 1920 X 1080 – 125 DPI
    • 3840 X 2160 – 150 DPI

Window states​

Use the following tests with your app to find compatibility issues with different window states:

  1. Launch the game your app supports, then test the app in full-screen (game window). Make sure everything is located where you want it.
  2. Change to windowed mode and make sure that everything is where you want it.

Window behavior when "out of focus"​

Overwolf only displays overlays in-game when the game is in focus. That means that whenever your game is alt+tabbed or minimized, no app window should appear.

To test this behavior:

  1. Launch the game your app supports. Make sure that the app launched correctly.
  2. Use Alt + Tab to get out of the game and make sure that your app window does not show on desktop.
  3. Alt+tab back to the game, the app should work normally.

Overlay​

In-game behavior​

Test preconditions: The user is now running the game your app supports.

  1. Open the app window with your designated hotkey and make sure the overlay appears properly.
  2. Test general app functionality in-game and ensure your features work.
  3. Press the app window in spots where clicks going through to the game would have an effect. For example, click an app button which covers an in-game menu and make sure your clicks only affect the app layer and never that menu.
  4. Any in-game browser links must be opened in Overwolf's browser. If you want to open them in user's default browser, prompt the user to let them know.

Hotkeys​

General functionality​

  1. Use the hotkey to launch the app and ensure it launches properly.

  2. Use the hotkey again to minimize your app.

  3. Change your app's designated hotkey and make sure the change appears in the app's settings.

  4. Use the changed hotkey to launch the app and ensure it launches properly.

  5. Use the changed hotkey again to minimize your app.

  6. Allow the user to change the hotkey combination from inside your app’s settings panel by automatically directing the user to the Overwolf settings page. Make sure that the new hotkey is updated automatically in your app without needing to re-launch the app. For more information about hotkeys, seemanifest.json and overwolf.settings API pages.

  7. Launch your app using the dock icon while in-game to verify that it provides functionality to the user.

Test mid-game installation​

Preconditions: Uninstall your app to perform the following test.

  1. While in-game, open the Overwolf Appstore and install the app.

  2. Use hotkey to launch the app and verify that it successfully opens.

  3. Use hotkey again to minimize the app.

  4. Change the designated hotkey and use it again to launch the app.

  5. Used the changed hotkey again to minimize the app.

  6. Whenever a user changes their hotkey, verify that your app changes both functionality and UI.

Desktop​

Desktop app behavior​

  1. Launch your app while in desktop, verify that the app window appears in 30 milliseconds or less. It is acceptable that the app displays a loader until it's ready for use.

  2. Test your app's desktop functionality and features.

  3. Close the Overwolf client and launch your app using the desktop icon.

  4. Verify that all windows are closed upon exit as well as all app-related processes in http://localhost:54284 (except those that are designed to be background processes).

Game specific apps​

If your app supports a specific game only, make sure it runs when you launch that game and doesn't for others.

Performance​

Memory & CPU Usage​

Preconditions: Open both the Overwolf task manager and the Windows task manager.

  • Use your app's basic functionality, ensure it doesn't hang, lag or stops responding.
  • Use the app while checking the Overwolf task manager. App memory usage should rise and fall. If it doesn't drop back down after rising, the app might have a memory leak.
  • Click the performance tab in Windows Task manager and watch for spikes in CPU, memory or network usage while using your app.

Dock icon size​

Dock icons should be .png format, sized at 256×256 pixels and less than 30KB. This is both greyscale and full color icons.

Internet access​

  1. Disconnect from the internet and verify your app still launches normally.

  2. When offline, make sure your app displays a check your internet connection type message.

  3. Make users aware that the requires an internet connection to function. It should be clarified for users in the app's window.

  4. Use the follow methods to detect if you app is offline:

Ads​

Follow the following steps to enable ads in your app:

  1. Open the app, then open the developer console (Ctrl + Shift + I).
  2. Press the Console tab, then paste localStorage.owAdTestAd = true into the console, and press Enter.
  3. Press F5 to refresh the window.
  4. Ads should appear in your app in the locations you designated.
  5. If no ads are displayed, delete the BrowserCache folder located at AppData\Local\Overwolf\BrowserCache. You can also check the Disable cache option in the Network tab of the devtools. Then, relaunch the app.

Testing Ad visibility​

  1. Launch your app.

  2. Go over your app screens and look for the ad containers. Verify that the ads appear properly.

Ad click functionality​

  1. Launch your app.

  2. Click on any element that is hiding ads and make sure the ads do not open and that your element functions as intended.

  3. Click on an ad, it should open the ad's link in a new browser window.

  4. Click on the ad 5 times, make sure five different browsers windows are launched without crashing Overwolf or harming your app's performance.

Ads are not hidden​

  1. Open the app, then open the developer console (Ctrl + Shift + I).
  2. Press the Console tab, and verify that no ad processes are visible if no ads are displayed on screen.

Ads remove and refresh​

Ads should close when your app window is minimized.

  1. Open http://localhost:54284/ in a new browser window.

  2. Launch your app, navigate to a screen where ads are presented.

  3. Close the app using a hotkey or manually.

  4. No ad process should be found in the localhost.

Pre-Release​

App's opk file doesn't warn for any viruses​

Before sending any opk file for approval, check that it is virus free. Use virustotal.com to check for potential issues. Any OPK which virustotal has warnings for will not be tested.

Automated testing​

If you want to use automated testing tools to test your app before submission, consider using ChromeDriver and selenium-webdriver / webdriverio.

Requirements​

Download chromedriver.exe and place it in a folder of your choice (i.e. C:/webdrivers), then add that folder to your system's PATH. Make sure your app is closed.

To add the folder to the PATH:

  1. Open the System Control Panel.
  2. Press System.
  3. Press Advanced System settings, go to the Advanced tab, then press Environment variables.
  4. In the System variables pane, select Environment Variables.
  5. Select PATH, then press Edit, and in the list of paths, pres New.
  6. In the empty line enter the path to your folder, then press OK and then OK again.

After you added the folder to the PATH, run the Overwolf client with the command line flag --enable-automation, then relaunch your app.

For the examples below: install and launch Replay HUD.

Testing using WebDriver with selenium-webdriver package:​

  1. Run npm install selenium-webdriver in your node project.
  2. Install and launch your app.
  3. Take note of your app's window title.
  4. Create a driver instance according to the following example:
// ReplayHUD automation with selenium-webdriver
const { Builder, By, Key, until } = require("selenium-webdriver");
const chrome = require("selenium-webdriver/chrome");

const options = new chrome.Options();
options.options_["debuggerAddress"] = "localhost:54284"; // Overwolf Remote Debugger

const xpath = "/html/body/ng-include/div/div[5]/div[2]/div[1]/article[4]/div";

(async function example() {
  let driver = await new Builder()
    .forBrowser("chrome")
    .setChromeOptions(options)
    .build();

  try {
    // Find and switch to your app's window
    const handles = await driver.getAllWindowHandles();
    for (let handle of handles) {
      await driver.switchTo().window(handle);
      if ((await driver.getTitle()) === "Replay HUD") {
        break;
      }
    }
    // Enjoy
    const element = await driver.wait(
      until.elementLocated(By.xpath(xpath)),
      30
    );
    await element.click();
  } finally {
    await driver.quit();
  }
})();

Testing using WebDriver with webdriverio in Standalone Mode:​

If you want to run webdriverio in Testrunner Mode, see Getting started. The configuration file (wdio.conf.js) includes the options used in the Standalone Mode example below.

  1. Run npm install webdriverio in your node project.
  2. Create a browser session according to the following example:
// ReplayHUD automation with webdriverio in Standalone Mode
const { remote } = require("webdriverio");

const xpath = "/html/body/ng-include/div/div[5]/div[2]/div[1]/article[4]/div";

(async () => {
  const browser = await remote({
    automationProtocol: "devtools",
    browserName: "chrome",
    capabilities: {
      "goog:chromeOptions": {
        debuggerAddress: "localhost:54284",
      },
    },
  });

  await browser.switchWindow("Replay HUD");

  const button = await browser.$(xpath);
  await button.waitForClickable({ timeout: 3000 });
  await button.click();
  await browser.deleteSession();
})();