# Overview

A quick overview for developers.

## Introduction

On this page, we will introduce some core concepts used throughout the platform. For a full overview of our [REST API](https://gitlab.com/neowit/docs/-/blob/master/gitbook/broken-reference/README.md)s, see the [API reference](/rest-api/api-reference).

## Organizations

An organization represents the main grouping entity within the Neowit platform. All other entities within the API are owned by an organization. An organization may have child organizations; supporting use-cases such as resellers, partners, distributors etc.

## Users

A user account represents a person and requires an email. Authentication is done using email and password or Single Sign-On (SAML). A user belongs to an organization and can have role of *Member* or *Administrator*. Currently, a *Member* is granted access to the Workspace application, while *Administrator*s have additional access to administrating Organizations, Users, Spaces, Integrations, Dashboards and Workspaces.

## Service accounts

A Service Account represents an machine account which is suitable for creating custom integrations. See [Service Accounts](https://gitlab.com/neowit/docs/-/blob/master/gitbook/broken-reference/README.md) for more details.

## Spaces

A Space represents entities of a building. A Space has a name, type and optional children; forming a hierarchical tree of spaces where the root would typically represent the building, the first level of children would represent the floor and second level typically represents objects on the floor such as rooms, desks and so on.

Each Space is given a [GeoJSON](https://geojson.org/) geometry, which is used to form the current 2D floorplans. Currently, we recommend using the editor in the App to control these Spaces. If you want to get your hands dirty with this, look at [Coordinate systems](/rest-api/coordinate-systems).

## Integrations

## Devices

Devices are managed by *Integrations,* and can represent a physical or virtual device. A device can provide a collection of *SensorType*s, denoting different types of metrics that can be ingested or queried. A Device may optionally be associated with a Space (e.g., temperature sensor in Room1), and can be given a [GeoJSON](https://geojson.org/) Point location to make it easier physically find the device.

## Workspaces

## Bookings


# Introduction

A quick introduction to our REST APIs and how to get started using it.

## Overview

Our REST APIs can be used to interact with our services programmatically. Our own web application is using the same APIs as documented here.

## First steps

The only thing you need to get started using our REST APIs is a [Service Account](https://gitlab.com/neowit/docs/-/blob/master/gitbook/rest-api/broken-reference/README.md) with sufficient permissions for the task you want to do. While subsequent Service Accounts can be managed through the REST API itself, an initial service account must be set up using the web application as an entry point.

## Start using our REST API

After [Creating service accounts](/service-accounts/creating-service-accounts), you may proceed to viewing the full [API reference](/rest-api/api-reference) to explore which API endpoints are available.

### Authentication

Our REST API supports two methods of authentication, both using a Service Account for access control.

* **Basic Auth:** Simple username and password authentication; for quick prototyping.
* **OAuth2:** A robust exchange of credentials for an access token used to authenticate.

We **strongly** recommend using the OAuth2 authentication flow when integrating with our REST API and to use Basic Auth for quick experimentation and exploration of our APIs.

Head over to our [Authentication](/rest-api/authentication) section to get started.


# Authentication

An overview of the two supported methods of authenticating to our REST APIs.

## OAuth2

We **strongly** recommend using the OAuth2 authentication flow when integrating with our REST API. It utilizes the [JWT](https://jwt.io/) as a medium of exchange and a [Service Account](/service-accounts/creating-service-accounts) for access control. New integrations should use `RS256` Service Account keys. After the authentication exchange is implemented, the returned access token allows you to perform HTTP requests efficiently.

Read about how to implement [OAuth2](/rest-api/authentication/oauth2) to get started.

## Basic Auth

We also support Basic Auth for authenticating with a Service Account. While this is not recommended for production-level integrations, it can be quite useful for quick experimentation and prototyping. It is disabled by default but can be enabled when creating or updating the [Service Account](/service-accounts/creating-service-accounts). Basic Auth only works with legacy `HS256` keys and stops working when the key expires.

Read about how to implement [Basic Auth](/rest-api/authentication/basic-auth) to get started.


# OAuth2

A guide on how to implement an OAuth2 flow for authenticating to our REST APIs.

## Overview

This guide explains how to authenticate a Service Account for REST API integrations. Our OAuth2 implementation is based on the [RFC7523](https://www.rfc-editor.org/rfc/rfc7523) JWT bearer grant flow.

The recommended signing algorithm is `RS256`. With `RS256`, you keep the RSA private key in your application and upload the matching public key when creating the Service Account key. `HS256` is still supported for legacy shared-secret integrations.

## Migration Guide

{% hint style="warning" %}
**Important:** The OAuth2 Service Account flow supports `RS256` and requires RFC7523-style JWT claims. Existing `HS256` integrations can continue to work, but new integrations should use `RS256`.
{% endhint %}

### Key Changes

1. **Algorithm support**
   * `RS256` is recommended for new Service Account keys.
   * `HS256` is supported for legacy shared-secret keys.
   * The JWT header `alg` must match the Service Account key type.
   * `RS256` keys cannot be used with Basic Auth.
2. **Key expiration**
   * Service Account keys can have an optional expiration date.
   * Expired keys cannot be used to exchange JWT assertions for access tokens.
   * Create a new key before the old key expires to avoid integration downtime.
3. **JWT claims**
   * `aud` (Audience): Must be set to the token endpoint URL (`https://app.neowit.io/api/auth/oauth/token`).
   * `iss` (Issuer): Must be set to your service account ID.
   * `sub` (Subject): Must be set to your service account ID.
   * `exp` (Expiration): Required.
   * `iat` (Issued at): Recommended.
   * `jti` (JWT ID): Recommended for replay protection. If supplied, the same `jti` cannot be reused before the JWT expires.
4. **Response body format**
   * The response uses snake\_case field names per OAuth2 specification.
   * Use `access_token` instead of `accessToken`.
   * `token_type` is `Bearer`.
   * `expires_in` is the remaining access-token lifetime in seconds.

### Migration Checklist

* [ ] Create an `RS256` Service Account key and store the private key securely.
* [ ] Set a key expiration date if your credential rotation policy requires one.
* [ ] Sign JWT assertions with `alg: "RS256"` and the private key.
* [ ] Include `kid`, `aud`, `iss`, `sub`, and `exp` in the JWT assertion.
* [ ] Parse `access_token`, `token_type`, and `expires_in` from the response.
* [ ] Refresh the access token before `expires_in` reaches zero.

## Prerequisites

A [Service Account](/service-accounts/creating-service-accounts) must be created in the organization before continuing. For `RS256`, the Service Account key must be created with a public RSA key, and your application must keep the corresponding private key. The key may also have an optional expiration date; after that date, token exchange with that key fails.

## Example

The example code in this page is provided as is. It may not work unchanged in your environment and should be used as a quick implementation guide rather than production-ready code.

### Environment Setup

{% tabs %}
{% tab title="Python" %}

```bash
pip install pyjwt cryptography requests
```

{% endtab %}

{% tab title="Node.js" %}

```bash
npm install jsonwebtoken axios
```

{% endtab %}

{% tab title="Go" %}

```bash
go get github.com/golang-jwt/jwt/v5
```

{% endtab %}
{% endtabs %}

### Source Code

{% tabs %}
{% tab title="Python" %}

```python
import time
from pathlib import Path

import jwt        # pip install pyjwt cryptography
import requests   # pip install requests

token_endpoint = 'https://app.neowit.io/api/auth/oauth/token'
api_url = 'https://app.neowit.io/api/space/v1/space'

# Store these outside the application source.
service_account_id = '<your service account id>'
service_account_key_id = '<your service account key id>'
private_key = Path('private_key.pem').read_text()


def get_access_token(account_id, key_id, signing_key):
    now = int(time.time())

    jwt_headers = {
        'alg': 'RS256',
        'kid': key_id,
    }

    jwt_payload = {
        'iat': now,
        'exp': now + 3600,
        'aud': token_endpoint,
        'iss': account_id,
        'sub': account_id,
        'jti': f'{account_id}-{now}',
    }

    encoded_jwt = jwt.encode(
        payload=jwt_payload,
        key=signing_key,
        algorithm='RS256',
        headers=jwt_headers,
    )

    response = requests.post(
        url=token_endpoint,
        headers={'Content-Type': 'application/x-www-form-urlencoded'},
        data={
            'assertion': encoded_jwt,
            'grant_type': 'urn:ietf:params:oauth:grant-type:jwt-bearer',
        },
    )

    if response.status_code != 200:
        print('Status Code: {}'.format(response.status_code))
        print(response.json())
        return None

    return response.json()


def main():
    auth = get_access_token(
        service_account_id,
        service_account_key_id,
        private_key,
    )
    if auth is None:
        return

    response = requests.get(
        url=api_url,
        headers={'Authorization': 'Bearer ' + auth['access_token']},
    )
    print(response.json())


if __name__ == '__main__':
    main()
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const fs = require('fs');
const jwt = require('jsonwebtoken');    // npm install jsonwebtoken
const axios = require('axios').default; // npm install axios

const tokenEndpoint = 'https://app.neowit.io/api/auth/oauth/token';
const apiURL = 'https://app.neowit.io/api/space/v1/space';

// Store these outside the application source.
const serviceAccountID = '<your service account id>';
const serviceAccountKeyID = '<your service account key id>';
const privateKey = fs.readFileSync('private_key.pem', 'utf8');

async function getAccessToken(accountID, keyID, signingKey) {
    const now = Math.floor(Date.now() / 1000);

    const jwtHeaders = {
        alg: 'RS256',
        kid: keyID,
    };

    const jwtPayload = {
        iat: now,
        exp: now + 3600,
        aud: tokenEndpoint,
        iss: accountID,
        sub: accountID,
        jti: `${accountID}-${now}`,
    };

    const assertion = jwt.sign(
        jwtPayload,
        signingKey,
        {
            algorithm: 'RS256',
            header: jwtHeaders,
        },
    );

    const body = new URLSearchParams({
        assertion,
        grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer',
    });

    const response = await axios.post(tokenEndpoint, body.toString(), {
        headers: {'Content-Type': 'application/x-www-form-urlencoded'},
    });

    return response.data;
}

async function main() {
    const auth = await getAccessToken(
        serviceAccountID,
        serviceAccountKeyID,
        privateKey,
    );

    const response = await axios.get(apiURL, {
        headers: {'Authorization': 'Bearer ' + auth.access_token},
    });

    console.log(JSON.stringify(response.data, null, 2));
}

main().catch((error) => {
    if (error.response) {
        console.error(error.response.data);
    }
    throw error;
});
```

{% endtab %}

{% tab title="Go" %}

```go
package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"net/url"
	"os"
	"strings"
	"time"

	jwt "github.com/golang-jwt/jwt/v5"
)

const (
	tokenEndpoint = "https://app.neowit.io/api/auth/oauth/token"
	apiURL        = "https://app.neowit.io/api/space/v1/space"

	serviceAccountID    = "<your service account id>"
	serviceAccountKeyID = "<your service account key id>"
)

type AuthResponse struct {
	AccessToken string `json:"access_token"`
	TokenType   string `json:"token_type"`
	ExpiresIn   int64  `json:"expires_in"`
}

func getAccessToken(ctx context.Context, accountID, keyID string, signingKey []byte) (*AuthResponse, error) {
	privateKey, err := jwt.ParseRSAPrivateKeyFromPEM(signingKey)
	if err != nil {
		return nil, fmt.Errorf("failed to parse private key: %w", err)
	}

	now := time.Now()
	claims := jwt.RegisteredClaims{
		ID:        fmt.Sprintf("%s-%d", accountID, now.Unix()),
		Issuer:    accountID,
		Subject:   accountID,
		Audience:  jwt.ClaimStrings{tokenEndpoint},
		IssuedAt:  jwt.NewNumericDate(now),
		ExpiresAt: jwt.NewNumericDate(now.Add(time.Hour)),
	}

	token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
	token.Header["kid"] = keyID

	assertion, err := token.SignedString(privateKey)
	if err != nil {
		return nil, fmt.Errorf("failed to sign JWT: %w", err)
	}

	body := url.Values{
		"assertion":  {assertion},
		"grant_type": {"urn:ietf:params:oauth:grant-type:jwt-bearer"},
	}.Encode()

	req, err := http.NewRequestWithContext(
		ctx,
		http.MethodPost,
		tokenEndpoint,
		strings.NewReader(body),
	)
	if err != nil {
		return nil, fmt.Errorf("failed to create request: %w", err)
	}
	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")

	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, fmt.Errorf("failed to perform request: %w", err)
	}
	defer res.Body.Close()

	if res.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("unexpected status code: %d", res.StatusCode)
	}

	var auth AuthResponse
	if err := json.NewDecoder(res.Body).Decode(&auth); err != nil {
		return nil, fmt.Errorf("failed to decode response: %w", err)
	}
	return &auth, nil
}

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
	defer cancel()

	privateKey, err := os.ReadFile("private_key.pem")
	if err != nil {
		log.Fatal(err)
	}

	auth, err := getAccessToken(ctx, serviceAccountID, serviceAccountKeyID, privateKey)
	if err != nil {
		log.Fatal(err)
	}

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, apiURL, nil)
	if err != nil {
		log.Fatal(err)
	}
	req.Header.Set("Authorization", "Bearer "+auth.AccessToken)

	res, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer res.Body.Close()

	var body map[string]interface{}
	if err := json.NewDecoder(res.Body).Decode(&body); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%#v\n", body)
}
```

{% endtab %}
{% endtabs %}

### HS256 Legacy Signing

For legacy `HS256` keys, sign with the Service Account key secret instead of an RSA private key. The JWT header must use `alg: "HS256"`, and the `kid` must point to an `HS256` key.

{% tabs %}
{% tab title="Python" %}

```python
encoded_jwt = jwt.encode(
    payload=jwt_payload,
    key=service_account_secret,
    algorithm='HS256',
    headers={'alg': 'HS256', 'kid': service_account_key_id},
)
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const assertion = jwt.sign(
    jwtPayload,
    serviceAccountSecret,
    {
        algorithm: 'HS256',
        header: {alg: 'HS256', kid: serviceAccountKeyID},
    },
);
```

{% endtab %}

{% tab title="Go" %}

```go
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
token.Header["kid"] = serviceAccountKeyID
assertion, err := token.SignedString([]byte(serviceAccountSecret))
```

{% endtab %}
{% endtabs %}

## Code Walkthrough

Authenticating a client is a three-step process:

1. Create and sign a JWT assertion.
2. Exchange the JWT assertion for an access token.
3. Use the access token to call the REST API.

### 1. Create The JWT

The JWT must contain a header with the signing algorithm and key ID:

```json
{
    "alg": "RS256",
    "kid": "<service account key id>"
}
```

The JWT payload must identify the Service Account and token endpoint:

```json
{
    "iat": 1719849600,
    "exp": 1719853200,
    "aud": "https://app.neowit.io/api/auth/oauth/token",
    "iss": "<service account id>",
    "sub": "<service account id>",
    "jti": "<unique assertion id>"
}
```

The `exp` claim is required. It must be in the future, expressed as a Unix timestamp in seconds. The OAuth2 endpoint signs the returned access token with the same expiration deadline, and the response `expires_in` value is calculated from that deadline.

Use a short `exp` value and refresh by creating a new JWT assertion. The examples use one hour. If you include a `jti`, generate a fresh value for every token exchange; replaying the same `jti` before expiration is rejected.

### 2. Exchange For Access Token

Send a `POST` request to `https://app.neowit.io/api/auth/oauth/token` with a Form URL-Encoded body:

1. `assertion`: The encoded JWT string.
2. `grant_type`: `urn:ietf:params:oauth:grant-type:jwt-bearer`.

The request must have `Content-Type: application/x-www-form-urlencoded`.

```
assertion=Base64Url(header).Base64Url(payload).Base64Url(signature)&grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer
```

The response has this format:

```json
{
    "access_token": "<access token for use in the Authorization header>",
    "token_type": "Bearer",
    "expires_in": 3600
}
```

`expires_in` is the number of seconds until the access token expires.

### 3. Access The REST API

Include the returned access token in the `Authorization` header for REST API calls:

```
Authorization: Bearer <access token>
```

## Refreshing The Access Token

Access tokens cannot be refreshed directly. To get a new access token, create and sign a new JWT assertion with a future `exp`, then repeat the token exchange. Refresh before `expires_in` reaches zero or when the API returns `401 Unauthorized`.

## Service Account Key Expiration

A Service Account key can also have its own optional expiration. If the key has expired, the OAuth2 token endpoint rejects JWT assertions that reference that key, even if the JWT assertion itself has a future `exp`.

Key expiration is configured when the key is created. In the REST API this is represented as `expiresAt`, a Unix timestamp in seconds. In the app, use the key expiration date field when creating the key.

To rotate credentials, create a new key before the current key expires, deploy the new private key or secret to your integration, confirm token exchange works, and then delete the old key.


# Basic Auth

A guide on how to use Basic Auth for authenticating to our REST APIs.

## Overview

Basic Auth is supported in most request libraries and is often as simple as adding a username- and password parameter. To get you up and running quickly, we present a few language-specific methods by fetching a list of projects available from the REST API using a [Service Account](/service-accounts/creating-service-accounts) for access control.

{% hint style="info" %}
We don't recommend using this authentication flow for production use, the main use for Basic Auth is for quick exploration and experimentation.
{% endhint %}

{% hint style="warning" %}
Basic Auth only works with legacy `HS256` Service Account keys that have a shared secret. It is not supported for `RS256` keys. If the key has an expiration date, Basic Auth stops working when the key expires.
{% endhint %}

## Prerequisites

A [Service Account](/service-accounts/creating-service-accounts) with a non-expired `HS256` key must be created in the organization before continuing.

## Code Sample

The following examples send a GET request to list available spaces in your organization. See the API Reference for all available API calls.

{% tabs %}
{% tab title="Python" %}
{% code lineNumbers="true" %}

```python
import requests  # pip install requests

# Inputs
key_id = '<service account key id>'
secret = '<service account secret>'

if __name__ == '__main__':
    # Send GET request to endpoint of choice with Basic Auth authentication.
    spaces = requests.get(
        url='https://app.neowit.io/api/space/v1/space',
        auth=(key_id, secret),
    )

    # Print response contents.
    print(spaces.json())
```

{% endcode %}
{% endtab %}

{% tab title="Node.js" %}
{% code lineNumbers="true" %}

```javascript
// modules
const axios = require('axios').default; // npm install axios

// Inputs
const keyID = '<service account key id>';
const secret = '<service account secret>';

async function main() {
    // Send GET request to endpoint of choice with Basic Auth authentication.
    const response = await axios({
        method: 'GET',
        url: 'https://app.neowit.io/api/space/v1/space',
        auth: {
            username: keyID,
            password: secret,
        },
    });

    // Print response contents.
    console.log(JSON.stringify(response.data, null, 2));
}
main();
```

{% endcode %}
{% endtab %}

{% tab title="Go" %}
{% code lineNumbers="true" %}

```go
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"time"
)

const (
	spacesURL = "https://app.neowit.io/api/space/v1/space"
	keyID     = "<service account key id>"
	secret    = "<service account secret>"
)

func main() {
	// Create a custom http Client with timeout.
	client := &http.Client{Timeout: time.Second * 3}

	// Create the request object with method, URL, but no optional body.
	req, err := http.NewRequest("GET", spacesURL, nil)
	if err != nil {
		log.Fatal(err)
	}

	// Set the request's Authorization header to use HTTP Basic Authentication.
	req.SetBasicAuth(keyID, secret)

	// Send an HTTP request and return an HTTP response.
	response, err := client.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer response.Body.Close()

	// Convert response body to map.
	var body map[string]interface{}
	if err = json.NewDecoder(response.Body).Decode(&body); err != nil {
		log.Fatal(err)
	}

	// Pretty print the response body.
	prettyBody, _ := json.MarshalIndent(body, "", "    ")
	fmt.Println(string(prettyBody))
}

```

{% endcode %}
{% endtab %}

{% tab title="cURL" %}

```bash
export NW_SERVICE_ACCOUNT_KEY_ID="<service account key id>"
export NW_SERVICE_ACCOUNT_SECRET="<service account secret>"

curl -X GET "https://app.neowit.io/api/space/v1/space" \
    -H "accept: application/json" \
    -u $NW_SERVICE_ACCOUNT_KEY_ID:$NW_SERVICE_ACCOUNT_SECRET
```

{% endtab %}
{% endtabs %}


# Error codes

Summary of common error codes returned by our REST API.

## Common response body

Most API endpoints return a common response body when an errors occurs. These responses can contain useful hints while developing or debugging, but is not designed to be directly shown to the user.

```json
{
  "reason": "invalid space",
  "status": "bad request",
  "traceID": "123wedt",
  "validations": {
    "user.email": "required"
  }
}
```

<table><thead><tr><th width="163.33333333333331">Field</th><th width="122">Type</th><th>Description</th></tr></thead><tbody><tr><td><code>reason</code></td><td><code>string</code></td><td>A summary of what when wrong.</td></tr><tr><td><code>status</code></td><td><code>string</code></td><td>Text version of the response code.</td></tr><tr><td><code>traceID</code></td><td><code>string</code></td><td>An id that may be supplied to support for internal debugging.</td></tr><tr><td><code>validations</code></td><td><code>object</code></td><td>Hints of fields that failed validating.</td></tr></tbody></table>

## 400 - Bad Request <a href="#id-400" id="id-400"></a>

This is most likely due to an invalid argument as a part of a path, query parameter, or request body. Please see the error message for more information.

## 401 - Unauthorized <a href="#id-401" id="id-401"></a>

The user or the Service Account can not be authenticated towards our API.

### Access Tokens

If you are using an Access Token received from our OAuth2 endpoint, please note that these are only valid for *one hour*. For uninterrupted operations using Access Tokens, make sure to get a new one before the previous one expires.

See our [Authentication guide](/rest-api/authentication) for more details.

## 403 - Forbidden <a href="#id-403" id="id-403"></a>

The user or service account does not have access to this resource.

## 404 - Not found <a href="#id-404" id="id-404"></a>

The requested resource could not be found. Make sure the URL is correct. Note that the resource may have been removed after it was discovered.

## 409 - Conflict <a href="#id-409" id="id-409"></a>

The resource you're trying to create already exists. This can happen for certain resources which have unique constraints (for example named tags).

## 500 - Internal server error <a href="#id-500" id="id-500"></a>

An internal error occurred in our services.

These may be intermittent, and in general we recommend re-trying with an exponential backoff with a minimum of 1-second delay. If your integration sends many requests our way, we also recommend adding jitter to this retry mechanism to stop thundering herds.

If these errors persist over a longer period, check our [Status page](https://neowit.statuspage.io/). If nothing is showing up there, get in touch with [Support](https://support.neowit.io/). Please include the `traceID` from the response body if this is present, as it can help support and our engineering team figure out what when wrong quicker.

## 503 - Service Unavailable <a href="#id-503" id="id-503"></a>

See [500 - Internal server error](#500)

## 504 - Gateway timeout <a href="#id-504" id="id-504"></a>

See [500 - Internal server error](#500)


# API reference

API reference overview

## Overview

Our REST APIs are documented using the [OpenAPI v2 specification](https://swagger.io/specification/v2/).

* [**Reference docs**](https://app.neowit.io/api/swagger/index.html)
* [**JSON reference**](https://app.neowit.io/api/swagger/doc.json)

The link to the reference docs are also available in the header of this site for quick access.


# Coordinate systems

An introduction to the coordinate system used for placement of spaces and devices.

Currently, we support two different coordinate systems for Spaces and one coordinate system for Devices.

* [Real coordinates](#real-coordinates) - Applicable to Spaces of type Building.
* [Virtual coordinates](#virtual-coordinates) - Applicable to Devices and Spaces that are not of type Building.

## Real coordinates

Real coordinates is only applicable Spaces of type Building, and represent the lat / lon location of the building. This is specified in the GeoJSON coordinates as \[latitude, longtitude, altitude], where the altitude is optional and ignored.

{% code title="Example Building Space" %}

```json
{
  ....
  "type": "SPACE_BUILDING",
  "coordinates": "COORDS_REAL",
  "image": "https://app.neowit.io/api/image/v1/image/<someimage>",
  "features": {
    "type": "Feature",
    "geometry": {
      "type": "Point",
      "coordinates": [
        -74.0088256,
        40.7060361,
        0
      ]
    },
    "properties": {
      "address": "Wall St, New York, NY, USA"
    }
  }
}
```

{% endcode %}

## Virtual coordinates

The virtual coordinate system is a bit difficult to work with, as it is tightly coupled with our web app and its [deck.gl](https://deck.gl/) usage. In the future, we will introduce a better coordinate system that is easier to work with and maps better to real word coordinates.

### Spaces

All Spaces except Buildings are defined using a GeoJSON Feature using Polygon geometry.

```json
{
    ...
    "name": "My Room",
    "type": "SPACE_ROOM",
    "coordinates": "COORDS_VIRTUAL",
    "image": "https://app.neowit.io/api/image/v1/image/<id>.png",
    "features": {
        "type": "Feature",
        "geometry": {
            "type": "Polygon",
            "coordinates": [
                [
                    [
                        796.5946119869824,
                        908.4491380427061
                    ],
                    [
                        796.5946119869824,
                        717.0977754704593
                    ],
                    [
                        987.9459745592292,
                        717.0977754704593
                    ],
                    [
                        987.9459745592292,
                        908.4491380427061
                    ],
                    [
                        796.5946119869824,
                        908.4491380427061
                    ]
                ]
            ]
        },
        "properties": {
            "shape": "Rectangle"
        }
    }
}
```

### Devices

Devices may be given a position using a GeoJSON Feature using Point geometry. Note that this only makes sense if the Device has an associated Space.

```json
{
    ...
    "name": "A Device",
    "spaceId": "<associatedSpaceId>",
    "features": {
        "type": "Feature",
        "geometry": {
            "type": "Point",
            "coordinates": [
                531.0645562037637,
                11.737089201877893
            ]
        },
        "properties": null
    }
}
```

### Basic workings of the coordinate system

* 2-dimensional [Cartesian](https://en.wikipedia.org/wiki/Cartesian_coordinate_system) coordinate system (x, y) - floating numbers.
* Bounding box is defined as with a range of (0, 0) - (1000, 1000). Which yields the following coordinates:

```
   top: left(0, 1000) right(1000, 1000)
bottom: left(0,    0) right(1000,    0)
```

* The image uploaded as part of the Floor is preserved in its full size, but we do an object fit into the bounding box, preserving its aspect ratio, when mapping it into the coordinate system. Origo of the object fit is Cartesian (x, y) = (0, 0).
* Coordinates outside of the \[0, 1000] range are permitted for historic reasons.

### Converting between image pixel and Cartesian

Example conversion code for TypeScript is show below. This is a small utility class to convert between Cartesian and pixel coordinates defined by the uploaded image.

{% code title="TypeScript conversion code" lineNumbers="true" %}

```typescript
type Size = {
	width: number;
	height: number;
};

type Point = {
	x: number;
	y: number;
};

type Domain = {
	max: number;
	min: number;
};

export class Convert {
	private readonly BOX_SIZE = 1000;

	private size: Size;

	private box: Size;

	constructor(size: Size) {
		this.size = size;
		this.box = this.pixelSizeToCartesian(size);
	}

	pixelPointToCartesian(pixel: Point): Point {
		const x = Convert.normalize(
			pixel.x,
			{min: 0, max: this.size.width},
			{min: 0, max: this.box.width},
		);
		const y = Convert.normalize(
			this.size.height - pixel.y,
			{min: 0, max: this.size.height},
			{min: 0, max: this.box.height},
		);
		return {x, y};
	}

	cartesianPointToPixel(coords: Point): Point {
		const x = Convert.normalize(
			coords.x,
			{min: 0, max: this.box.width},
			{min: 0, max: this.size.width},
		);
		const y =
			this.size.height -
			Convert.normalize(
				coords.y,
				{min: 0, max: this.box.height},
				{min: 0, max: this.size.height},
			);
		return {x, y};
	}

	pixelSizeToCartesian(size: Size): Size {
		const ratio = size.height / size.width;
		const width = ratio >= 1 ? this.BOX_SIZE / ratio : this.BOX_SIZE;
		const height = ratio >= 1 ? this.BOX_SIZE : this.BOX_SIZE * ratio;
		return {width, height};
	}

	private static normalize(
		input: number,
		inDomain: Domain,
		outDomain: Domain,
	): number {
		const inputRange = inDomain.max - inDomain.min;
		const outputRange = outDomain.max - outDomain.min;
		return (
			((input - inDomain.min) / inputRange) * outputRange + outDomain.min
		);
	}
}
```

{% endcode %}

#### Example 1 - Image where width > height

```typescript
// Image size (WxH) = (768x576)
// .----------------------------------------------------.
// | orientation  | image pixel (x,y) | cartesian (x,y) |
// |--------------|-------------------|-----------------|
// | left bottom  |   (  0, 576)      | (   0,   0)     |
// | left top     |   (  0,   0)      | (   0, 750)     |
// | right bottom |   (768, 576)      | (1000,   0)     | 
// | right top    |   (768,   0)      | (1000, 750)     |
// .----------------------------------------------------.
const c = new Convert({width: 768, height: 576});
console.log(
	'left  bottom PIXEL(  0,576) => (   0,   0)',
	c.cartesianPointToPixel({x: 0, y: 0}),
	c.pixelPointToCartesian({x: 0, y: 576}),
);
console.log(
	'left  top    PIXEL(  0,  0) => (   0, 750)',
	c.cartesianPointToPixel({x: 0, y: 750}),
	c.pixelPointToCartesian({x: 0, y: 0}),
);
console.log(
	'right bottom PIXEL(768,576) => (1000,   0)',
	c.cartesianPointToPixel({x: 1000, y: 0}),
	c.pixelPointToCartesian({x: 768, y: 576}),
);
console.log(
	'right top    PIXEL(768,  0) => (1000, 750)',
	c.cartesianPointToPixel({x: 1000, y: 750}),
	c.pixelPointToCartesian({x: 768, y: 0}),
);
```

#### Example 2 - Image where width < height

```typescript
// Image size (WxH) = (576x768)
// .----------------------------------------------------.
// | orientation  | image pixel (x,y) | cartesian (x,y) |
// |--------------|-------------------|-----------------|
// | left bottom  |   (  0, 768)      | (  0,    0)     |
// | left top     |   (  0,   0)      | (  0, 1000)     |
// | right bottom |   (576, 768)      | (750,    0)     |
// | right top    |   (576,   0)      | (750, 1000)     |
// .----------------------------------------------------.
const c = new Convert({width: 576, height: 768});
console.log(
	'left  bottom PIXEL(  0,768) => (  0,    0)',
	c.cartesianPointToPixel({x: 0, y: 0}),
	c.pixelPointToCartesian({x: 0, y: 768}),
);
console.log(
	'left  top    PIXEL(  0,  0) => (  0, 1000)',
	c.cartesianPointToPixel({x: 0, y: 1000}),
	c.pixelPointToCartesian({x: 0, y: 0}),
);
console.log(
	'right bottom PIXEL(576,768) => (750,    0)',
	c.cartesianPointToPixel({x: 750, y: 0}),
	c.pixelPointToCartesian({x: 576, y: 768}),
);
console.log(
	'right top    PIXEL(576,  0) => (750, 1000)',
	c.cartesianPointToPixel({x: 750, y: 1000}),
	c.pixelPointToCartesian({x: 576, y: 0}),
);
```

#### Example 3 - Image where width == height

```typescript
// Image size (WxH) = (768x768)
// .----------------------------------------------------.
// | orientation  | image pixel (x,y) | cartesian (x,y) |
// |--------------|-------------------|-----------------|
// | left bottom  |   (  0, 768)      | (   0,    0)    |
// | left top     |   (  0,   0)      | (   0, 1000)    |
// | right bottom |   (768, 768)      | (1000,    0)    |
// | right top    |   (768,   0)      | (1000, 1000)    |
// .----------------------------------------------------.
const c = new Convert({width: 768, height: 768});
console.log(
	'left  bottom PIXEL(  0,768) => (   0,    0)',
	c.cartesianPointToPixel({x: 0, y: 0}),
	c.pixelPointToCartesian({x: 0, y: 768}),
);
console.log(
	'left  top    PIXEL(  0,  0) => (   0, 1000)',
	c.cartesianPointToPixel({x: 0, y: 1000}),
	c.pixelPointToCartesian({x: 0, y: 0}),
);
console.log(
	'right bottom PIXEL(768,768) => (1000,    0)',
	c.cartesianPointToPixel({x: 1000, y: 0}),
	c.pixelPointToCartesian({x: 768, y: 768}),
);
console.log(
	'right top    PIXEL(768,  0) => (1000, 1000)',
	c.cartesianPointToPixel({x: 1000, y: 1000}),
	c.pixelPointToCartesian({x: 768, y: 0}),
);
```


# Query Language

The Neowit Query Language is a simple expression language for computing functions of the sensor data saved in the Neowit Platform. The following example averages a set of sensors and filters it to include only business hours.

```
businesshours(avg(floor3))
```

## Introduction

Neowit stores data collected from devices as time series of sensor data. Each sensor series has an associated sensor type which describes the type of data collected (e.g. Temperature). Because a device can collect data from multiple sensors, a device record is a set of series for each sensor type collected from the device. An expression typically computes a function of these records and returns another record. A record can contain data from multiple devices and thus contain multiple time series with the same sensor type as long as they belong to different devices.

## Evaluation Context

Each expression is evaluated in an implicit device context, which consists of identifiers mapping to records and the interval and resolution in the request.

#### Inputs

Inputs are the identifiers in the expression that resolve to the records being computed on. For example, the input variable `floor3` may resolve to a record of many devices and sensor data series.

* **Explicit Inputs** are identifiers defined in the request associated with the query expression. They can resolve to individual devices and sensors, or they can in turn resolve to other identifiers transitively.
* **Implicit Inputs** are identifiers that are available in every request. They are typically generated by Neowit from metadata gathered from the platform. For example, the `floor3` identifier may be generated from a space in the Neowit platform and refer to all sensor data from devices contained within that space.

#### Intervals

Intervals contain the time range requested (e.g. all of last week) as well as an associated resolution (e.g. hourly). When using aggregation functions, the resolution will be used as the target for which to aggregate or interpolate to. For example, sensor data is typically collected in second resolution, but when calling `avg(..)` with a requested hourly resolution, the data will be average by the hour. The resulting record will always have the same resolution as requested. If no aggregation was performed in the expression, Neowit will auto-aggregate the resulting record to the target resolution.

### Data Types

Each function takes a set of parameters with statically checked data types and has a return data type. A valid expression must always return a record data type, but sub-expressions may return other data types. For example, the function `hours(floor3, 9, 17)` returns the data in `floor3` filtered to only include data for hours between 9 and 17 (for each day). The type signature of the function is `hours(rec: record, hoursIndexStart: number, hoursIndexEnd: number): record` . The data types available are loosely inspired by JSON.

* **Number**: Any integer or floating point number.
* **String**
* **Boolean**
* **List**. Lists may take on any type but they must be homogeneous, i.e. every list item must have the same type, e.g. a list of strings.
* **Series**. An individual time series for a `(deviceId, sensorId)` pair.
* **Record**. A set of series.

#### Backtick Identifiers

Identifiers can be written either as a standard C-like identifier, e.g. `floor3_25` or they can be written using backticks which allows them to be represented with arbitrary strings containing whitespace etc, e.g. `` `Floor 123` ``

#### Identifier Lists

A list of identifier, e.g. `[floor2, floor3]` are automatically resolved to a record containing the data from all the identifiers in the list with an implicit `combine(..)` function applied to them that aggregates records row-wise.

### Function Overloading

Types are never annotated in an expression, but are inferred by the evaluated and statically checked. Each function (and binary operator) may have several overloaded type signatures allowing for inputs of different types to be used with the same function or operator.

#### Default Values

Some of the parameters to a function have default values making them optional. For example, in the function `avg(rec: record, fillna: boolean): record` the `fillna` parameter has a default value of `false` so that we can simply write `avg(floor3`).

#### Allowed Values

Some parameters may have a predefined set of allowed values (effectively enums). For example, in the function `agg(rec: record, method: string, fillna: boolean): record` The parameter `method` which specifies the aggregation function to use has a set of allowed values `auto`, `avg` ,`max` , etc.

### Binary Operators

In addition to function, expressions can also contain binary operators. These may be either logical or arithmetic. Binary operators are overloaded similar to functions so they work with a range of data types.

#### Logical Operators

These typically produce records of indicator values (1s or 0s) for when the expression is true. For example, the expression `floor3 < 5` returns a record of indicator values of the input record for when the values are less than 5. `floor2 > floor3` produces a record of indicator values for when the sensor data of `floor2` is greater than the corresponding values of `floor3`.

#### Arithmetic Operators

Similarly, arithmetic operators allow for basic arithmetic on various data types, e.g. `floor3 + floor2` will add the data from corresponding sensors in the operand records. `3 * floor3` will multiply all the values in the record.

[Query Language Reference](/rest-api/query-language/query-language-reference)

####


# Query Language Reference

## Functions

### agg

**agg(record: record, method: string, fillna: boolean): record**\
Aggregate a record

| Parameter | Type    | Description                                                      | Allowed Values                                  | Default |
| --------- | ------- | ---------------------------------------------------------------- | ----------------------------------------------- | ------- |
| record    | record  | The input record to aggregate                                    |                                                 |         |
| method    | string  | Aggreagation function to apply                                   | "auto", "avg", "lastValue", "max", "min", "sum" |         |
| fillna    | boolean | Interpolate the input record with last values before aggregating |                                                 | false   |

### avg

**avg(record: record, fillna: boolean): record**\
Average a record

| Parameter | Type    | Description                                                    | Allowed Values | Default |
| --------- | ------- | -------------------------------------------------------------- | -------------- | ------- |
| record    | record  | The input record to average                                    |                |         |
| fillna    | boolean | Interpolate the input record with last values before averaging |                | false   |

### businessdays

**businessdays(record: record): record**\
Filter a record to include only business days

| Parameter | Type   | Description                                  | Allowed Values | Default |
| --------- | ------ | -------------------------------------------- | -------------- | ------- |
| record    | record | The input record to filter for business days |                |         |

### businesshours

**businesshours(record: record): record**\
Filter a record to include only business hours

| Parameter | Type   | Description                                   | Allowed Values | Default |
| --------- | ------ | --------------------------------------------- | -------------- | ------- |
| record    | record | The input record to filter for business hours |                |         |

### clipday

**clipday(record: record, secondOffsetFromMidnightStart: number, secondOffsetFromMidnightEnd: number): record**\
Filter a record to include only the specified range of seconds offset from midnight

| Parameter                     | Type   | Description                                                                       | Allowed Values | Default |
| ----------------------------- | ------ | --------------------------------------------------------------------------------- | -------------- | ------- |
| record                        | record | The input record to clip                                                          |                |         |
| secondOffsetFromMidnightStart | number | Number of seconds offset from midnight at the start of each day to start clipping |                |         |
| secondOffsetFromMidnightEnd   | number | Number of seconds offset from midnight at the end of each day to end clipping     |                |         |

### combine

**combine(record: record, method: string, fillna: boolean, neverFillEmptyRows: boolean): record**\
Combine equal sensor types in a record with the specified method

| Parameter          | Type    | Description                                                | Allowed Values                                  | Default |
| ------------------ | ------- | ---------------------------------------------------------- | ----------------------------------------------- | ------- |
| record             | record  | The input record to combine                                |                                                 |         |
| method             | string  | Aggeregation strategy to use when combining sensor types   | "auto", "avg", "lastValue", "max", "min", "sum" |         |
| fillna             | boolean | Interpolate input record with last values before combining |                                                 | false   |
| neverFillEmptyRows | boolean |                                                            |                                                 | false   |

### combine2

**combine2(recordX: record, recordY: record, method: string, fillna: boolean, neverFillEmptyRows: boolean): record**\
Combine equal sensor types from two records with the specified method

| Parameter          | Type    | Description                                                | Allowed Values                                  | Default |
| ------------------ | ------- | ---------------------------------------------------------- | ----------------------------------------------- | ------- |
| recordX            | record  | First input record to combine                              |                                                 |         |
| recordY            | record  | Second input record to combine                             |                                                 |         |
| method             | string  | Interpolate input record with last values before combining | "auto", "avg", "lastValue", "max", "min", "sum" |         |
| fillna             | boolean | Interpolate input record with last values before combining |                                                 | false   |
| neverFillEmptyRows | boolean |                                                            |                                                 | false   |

### dayofweek

**dayofweek(): series**\
Weekday numbers (monday = 0) for the implicit range

### diff

**diff(record: record): record**\
The discrete difference for a record

| Parameter | Type   | Description                       | Allowed Values | Default |
| --------- | ------ | --------------------------------- | -------------- | ------- |
| record    | record | The input record to differentiate |                |         |

### fillna

**fillna(record: record): record**\
Fill NA/NaN values by propagating last valid observation forward to next valid

| Parameter | Type   | Description                               | Allowed Values | Default |
| --------- | ------ | ----------------------------------------- | -------------- | ------- |
| record    | record | The input record to fill with last values |                |         |

### get

**get(deviceId: string, sensors: list\[string], includes: list\[string], excludes: list\[string]): record**\
\[DEPRECATED] Get a single device record by using device and sensor IDs as strings

| Parameter | Type          | Description                                                                     | Allowed Values | Default |
| --------- | ------------- | ------------------------------------------------------------------------------- | -------------- | ------- |
| deviceId  | string        | Device ID to retrieve                                                           |                |         |
| sensors   | list\[string] | List of sensor IDs to retrieve, or empty to select all                          |                | \[]     |
| includes  | list\[string] | List of tags to use as include filter (devices must match all include tags)     |                | \[]     |
| excludes  | list\[string] | List of tags to use as exclude filter (devices must not match any exclude tags) |                | \[]     |

**get(deviceIds: list\[string], sensors: list\[string], includes: list\[string], excludes: list\[string]): record**\
\[DEPRECATED\[ Get a single record of multiple devices

| Parameter | Type          | Description                                                                     | Allowed Values | Default |
| --------- | ------------- | ------------------------------------------------------------------------------- | -------------- | ------- |
| deviceIds | list\[string] | List of Device IDs to retrieve                                                  |                |         |
| sensors   | list\[string] | List of sensor IDs to retrieve, or empty to select all                          |                | \[]     |
| includes  | list\[string] | List of tags to use as include filter (devices must match all include tags)     |                | \[]     |
| excludes  | list\[string] | List of tags to use as exclude filter (devices must not match any exclude tags) |                | \[]     |

### hours

**hours(record: record, hoursIndexStart: number, hoursIndexEnd: number): record**\
Filter a record to include only the specified hour range

| Parameter       | Type   | Description                        | Allowed Values | Default |
| --------------- | ------ | ---------------------------------- | -------------- | ------- |
| record          | record | The input record to filter         |                |         |
| hoursIndexStart | number | Index of the starting hour \[0-24] |                |         |
| hoursIndexEnd   | number | Index of the end hour \[0-24]      |                |         |

### linger

**linger(record: record, duration: number): record**\
Transform a record by delaying each transition to zero for up to `duration` units of the record resolution

| Parameter | Type   | Description                              | Allowed Values | Default |
| --------- | ------ | ---------------------------------------- | -------------- | ------- |
| record    | record | The input record to linger               |                |         |
| duration  | number | Number of resolution units to linger for |                |         |

### max

**max(record: record, fillna: boolean): record**\
Aggregate by taking the maxium value for each sub interval

| Parameter | Type    | Description                                             | Allowed Values | Default |
| --------- | ------- | ------------------------------------------------------- | -------------- | ------- |
| record    | record  | The input record to max over                            |                |         |
| fillna    | boolean | Interpolate input record with last values before maxing |                | false   |

### min

**min(record: record, fillna: boolean): record**\
Aggregate by taking the minimum value for each sub interval

| Parameter | Type    | Description                                             | Allowed Values | Default |
| --------- | ------- | ------------------------------------------------------- | -------------- | ------- |
| record    | record  | The input record to min over                            |                |         |
| fillna    | boolean | Interpolate input record with last values before mining |                | false   |

### reduce

**reduce(record: record, fillna: boolean): record**\
Reduce a record by adding up all the resolution intervals, keeping only the last resolution interval

| Parameter | Type    | Description                                               | Allowed Values | Default |
| --------- | ------- | --------------------------------------------------------- | -------------- | ------- |
| record    | record  | The input record to reduce                                |                |         |
| fillna    | boolean | Interpolate input record with last values before reducing |                | false   |

### sin

**sin(x: number): number**\
The sine of a number (used mainly for testing)

| Parameter | Type   | Description | Allowed Values | Default |
| --------- | ------ | ----------- | -------------- | ------- |
| x         | number |             |                |         |

**sin(series: series): series**\
The sine of a series (used mainly for testing)

| Parameter | Type   | Description | Allowed Values | Default |
| --------- | ------ | ----------- | -------------- | ------- |
| series    | series |             |                |         |

### sum

**sum(record: record, fillna: boolean): record**\
Aggregate a record using addition

| Parameter | Type    | Description                                              | Allowed Values | Default |
| --------- | ------- | -------------------------------------------------------- | -------------- | ------- |
| record    | record  | The input record to sum                                  |                |         |
| fillna    | boolean | Interpolate input record with last values before summing |                | false   |

### weekdays

**weekdays(record: record, weekdayIndices: list\[number]): record**\
Filter a record to include days from the given list of weekday indices

| Parameter      | Type          | Description                               | Allowed Values | Default |
| -------------- | ------------- | ----------------------------------------- | -------------- | ------- |
| record         | record        | The input record to filter                |                |         |
| weekdayIndices | list\[number] | List of weekday indices to include \[0-7] |                |         |

### zeros

**zeros(): series**\
A series of all zeros

## Binary Operators

| Operator | Left Operand Type | Right Operand Type | Result Type | Description                                                                                                                                                                                                  |
| -------- | ----------------- | ------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| <        | number            | number             | number      | Whether the first argument is less than the second argument                                                                                                                                                  |
| <        | series            | number             | series      | Yields a series of indicator values for when the series' values are less than the scalar argument                                                                                                            |
| <        | number            | series             | series      | Yields a series of indicator values for when the scalar argument is less than the series' values                                                                                                             |
| <        | series            | series             | series      | Yields a series of indicator values for when the first series' values are less than the corresponding value in the second series, omitting points absent from either series                                  |
| <        | record            | number             | record      | Yields a record of indicator values for when the record's values are less than the scalar argument                                                                                                           |
| <        | number            | record             | record      | Yields a record of indicator values for when the scalar argument is less than the record's values                                                                                                            |
| <        | record            | series             | record      | Yields a record of indicator values for when the record's values are less than the corresponding value in the series, omitting points absent from either the record or series                                |
| <        | series            | record             | record      | Yields a record of indicator values for when the series' values are less than the corresponding value in each of the record's series, omitting points absent from either the record or series                |
| <        | record            | record             | record      | Yields a record of indicator values for when the first record' values are less than the corresponding value in the second record, omitting fields and points absent from either record                       |
| <=       | number            | number             | number      | Whether the first argument is less than or equal to the second argument                                                                                                                                      |
| <=       | series            | number             | series      | Yields a series of indicator values for when the series' values are less than or equal to the scalar argument                                                                                                |
| <=       | number            | series             | series      | Yields a series of indicator values for when the scalar argument is less than or equal to the series' values                                                                                                 |
| <=       | series            | series             | series      | Yields a series of indicator values for when the first series' values are less than or equal to the corresponding value in the second series, omitting points absent from either series                      |
| <=       | record            | number             | record      | Yields a record of indicator values for when the record's values are less than or equal to the scalar argument                                                                                               |
| <=       | number            | record             | record      | Yields a record of indicator values for when the scalar argument is less than or equal to the record's values                                                                                                |
| <=       | record            | series             | record      | Yields a record of indicator values for when the record's values are less than or equal to the corresponding value in the series, omitting points absent from either the record or series                    |
| <=       | series            | record             | record      | Yields a record of indicator values for when the series' values are less than or equal to the corresponding value in each of the record's series, omitting points absent from either the record or series    |
| <=       | record            | record             | record      | Yields a record of indicator values for when the first record' values are less than or equal to the corresponding value in the second record, omitting fields and points absent from either record           |
| >        | number            | number             | number      | Whether the first argument is greater than the second argument                                                                                                                                               |
| >        | series            | number             | series      | Yields a series of indicator values for when the series' values are greater than the scalar argument                                                                                                         |
| >        | number            | series             | series      | Yields a series of indicator values for when the scalar argument is greater than the series' values                                                                                                          |
| >        | series            | series             | series      | Yields a series of indicator values for when the first series' values are greater than the corresponding value in the second series, omitting points absent from either series                               |
| >        | record            | number             | record      | Yields a record of indicator values for when the record's values are greater than the scalar argument                                                                                                        |
| >        | number            | record             | record      | Yields a record of indicator values for when the scalar argument is greater than the record's values                                                                                                         |
| >        | record            | series             | record      | Yields a record of indicator values for when the record's values are greater than the corresponding value in the series, omitting points absent from either the record or series                             |
| >        | series            | record             | record      | Yields a record of indicator values for when the series' values are greater than the corresponding value in each of the record's series, omitting points absent from either the record or series             |
| >        | record            | record             | record      | Yields a record of indicator values for when the first record' values are greater than the corresponding value in the second record, omitting fields and points absent from either record                    |
| >=       | number            | number             | number      | Whether the first argument is greater than or equal to the second argument                                                                                                                                   |
| >=       | series            | number             | series      | Yields a series of indicator values for when the series' values are greater than or equal to the scalar argument                                                                                             |
| >=       | number            | series             | series      | Yields a series of indicator values for when the scalar argument is greater than or equal to the series' values                                                                                              |
| >=       | series            | series             | series      | Yields a series of indicator values for when the first series' values are greater than or equal to the corresponding value in the second series, omitting points absent from either series                   |
| >=       | record            | number             | record      | Yields a record of indicator values for when the record's values are greater than or equal to the scalar argument                                                                                            |
| >=       | number            | record             | record      | Yields a record of indicator values for when the scalar argument is greater than or equal to the record's values                                                                                             |
| >=       | record            | series             | record      | Yields a record of indicator values for when the record's values are greater than or equal to the corresponding value in the series, omitting points absent from either the record or series                 |
| >=       | series            | record             | record      | Yields a record of indicator values for when the series' values are greater than or equal to the corresponding value in each of the record's series, omitting points absent from either the record or series |
| >=       | record            | record             | record      | Yields a record of indicator values for when the first record' values are greater than or equal to the corresponding value in the second record, omitting fields and points absent from either record        |
| ==       | number            | number             | number      | Whether the first argument is equal to the second argument                                                                                                                                                   |
| ==       | series            | number             | series      | Yields a series of indicator values for when the series' values are equal to the scalar argument                                                                                                             |
| ==       | number            | series             | series      | Yields a series of indicator values for when the scalar argument is equal to the series' values                                                                                                              |
| ==       | series            | series             | series      | Yields a series of indicator values for when the first series' values are equal to the corresponding value in the second series, omitting points absent from either series                                   |
| ==       | record            | number             | record      | Yields a record of indicator values for when the record's values are equal to the scalar argument                                                                                                            |
| ==       | number            | record             | record      | Yields a record of indicator values for when the scalar argument is equal to the record's values                                                                                                             |
| ==       | record            | series             | record      | Yields a record of indicator values for when the record's values are equal to the corresponding value in the series, omitting points absent from either the record or series                                 |
| ==       | series            | record             | record      | Yields a record of indicator values for when the series' values are equal to the corresponding value in each of the record's series, omitting points absent from either the record or series                 |
| ==       | record            | record             | record      | Yields a record of indicator values for when the first record' values are equal to the corresponding value in the second record, omitting fields and points absent from either record                        |
| !=       | number            | number             | number      | Whether the first argument is not equal to the second argument                                                                                                                                               |
| !=       | series            | number             | series      | Yields a series of indicator values for when the series' values are not equal to the scalar argument                                                                                                         |
| !=       | number            | series             | series      | Yields a series of indicator values for when the scalar argument is not equal to the series' values                                                                                                          |
| !=       | series            | series             | series      | Yields a series of indicator values for when the first series' values are not equal to the corresponding value in the second series, omitting points absent from either series                               |
| !=       | record            | number             | record      | Yields a record of indicator values for when the record's values are not equal to the scalar argument                                                                                                        |
| !=       | number            | record             | record      | Yields a record of indicator values for when the scalar argument is not equal to the record's values                                                                                                         |
| !=       | record            | series             | record      | Yields a record of indicator values for when the record's values are not equal to the corresponding value in the series, omitting points absent from either the record or series                             |
| !=       | series            | record             | record      | Yields a record of indicator values for when the series' values are not equal to the corresponding value in each of the record's series, omitting points absent from either the record or series             |
| !=       | record            | record             | record      | Yields a record of indicator values for when the first record' values are not equal to the corresponding value in the second record, omitting fields and points absent from either record                    |
| +        | number            | number             | number      | Adds the first argument with the second argument                                                                                                                                                             |
| +        | series            | number             | series      | Adds the scalar argument to the series' values and yields a series with the result                                                                                                                           |
| +        | number            | series             | series      | Adds the series' values to the scalar argument and yields a series with the result                                                                                                                           |
| +        | series            | series             | series      | Adds the first series' values to the correspondong value in the second series and yields a series with the result, omitting points absent from either series                                                 |
| +        | record            | number             | record      | Adds the scalar argument to the record's values and yields a record with the result                                                                                                                          |
| +        | number            | record             | record      | Adds the record's values to the scalar argument and yields a record with the result                                                                                                                          |
| +        | record            | series             | record      | Adds each record series' values to the correspondong value in the series and yields a record with the result, omitting points absent from either the record or series                                        |
| +        | series            | record             | record      | Adds the series' values to the correspondong value in each of the record's series and yields a record with the result, omitting points absent from either the record or series                               |
| +        | record            | record             | record      | Adds the first record's values to the correspondong value in the second record and yields a record with the result, omitting fields and points absent from either record                                     |
| -        | number            | number             | number      | Subtracts the second argument from the first argument                                                                                                                                                        |
| -        | series            | number             | series      | Subtracts the scalar argument from the series' values and yields a series with the result                                                                                                                    |
| -        | number            | series             | series      | Subtracts the series' values from the scalar argument and yields a series with the result                                                                                                                    |
| -        | series            | series             | series      | Subtracts the second series' values from the correspondong value in the first series and yields a series with the result, omitting points absent from either series                                          |
| -        | record            | number             | record      | Subtracts the scalar argument from the record's values and yields a record with the result                                                                                                                   |
| -        | number            | record             | record      | Subtracts the record's values from the scalar argument and yields a record with the result                                                                                                                   |
| -        | record            | series             | record      | Subtracts the series' values from the correspondong value in each of the record's series and yields a record with the result, omitting points absent from either the record or series                        |
| -        | series            | record             | record      | Subtracts each record series' values from the correspondong value in the series and yields a record with the result, omitting points absent from either the record or series                                 |
| -        | record            | record             | record      | Subtracts the second record's values from the correspondong value in the first record and yields a record with the result, omitting fields and points absent from either record                              |
| \*       | number            | number             | number      | Multiplies the first argument with the second argument                                                                                                                                                       |
| \*       | series            | number             | series      | Multiplies the series' values with the scalar argument and yields a series with the result                                                                                                                   |
| \*       | number            | series             | series      | Multiplies the scalar argument with the series' values and yields a series with the result                                                                                                                   |
| \*       | series            | series             | series      | Multiplies the first series' values with the correspondong value in the second series and yields a series with the result, omitting points absent from either series                                         |
| \*       | record            | number             | record      | Multiplies the record's values with the scalar argument and yields a record with the result                                                                                                                  |
| \*       | number            | record             | record      | Multiplies the scalar argument with the record's values and yields a record with the result                                                                                                                  |
| \*       | record            | series             | record      | Multiplies each record series' values with the correspondong value in the series and yields a record with the result, omitting points absent from either the record or series                                |
| \*       | series            | record             | record      | Multiplies the series' values with the correspondong value in each of the record's series and yields a record with the result, omitting points absent from either the record or series                       |
| \*       | record            | record             | record      | Multiplies the first record's values with the correspondong value in the second record and yields a record with the result, omitting fields and points absent from either record                             |
| /        | number            | number             | number      | Divides the first argument by the second argument                                                                                                                                                            |
| /        | series            | number             | series      | Divides the series' values by the scalar argument and yields a series with the result                                                                                                                        |
| /        | number            | series             | series      | Divides the scalar argument by the series' values and yields a series with the result                                                                                                                        |
| /        | series            | series             | series      | Divides the first series' values by the correspondong value in the second series and yields a series with the result, omitting points absent from either series                                              |
| /        | record            | number             | record      | Divides the record's values by the scalar argument and yields a record with the result                                                                                                                       |
| /        | number            | record             | record      | Divides the scalar argument by the record's values and yields a record with the result                                                                                                                       |
| /        | record            | series             | record      | Divides each record series' values by the correspondong value in the series and yields a record with the result, omitting points absent from either the record or series                                     |
| /        | series            | record             | record      | Divides the series' values by the correspondong value in each of the record's series and yields a record with the result, omitting points absent from either the record or series                            |
| /        | record            | record             | record      | Divides the first record's values by the correspondong value in the second record and yields a record with the result, omitting fields and points absent from either record                                  |
| %        | number            | number             | number      | Computes the modulo of the first argument with the second argument                                                                                                                                           |
| %        | series            | number             | series      | Computes the modulo of the series' values with the scalar argument and yields a series with the result                                                                                                       |
| %        | number            | series             | series      | Computes the modulo of the scalar argument with the series' values and yields a series with the result                                                                                                       |
| %        | series            | series             | series      | Computes the modulo of the first series' values with the correspondong value in the second series and yields a series with the result, omitting points absent from either series                             |
| %        | record            | number             | record      | Computes the modulo of the record's values with the scalar argument and yields a record with the result                                                                                                      |
| %        | number            | record             | record      | Computes the modulo of the scalar argument with the record's values and yields a record with the result                                                                                                      |
| %        | record            | series             | record      | Computes the modulo of each record series' values with the correspondong value in the series and yields a record with the result, omitting points absent from either the record or series                    |
| %        | series            | record             | record      | Computes the modulo of the series' values with the correspondong value in each of the record's series and yields a record with the result, omitting points absent from either the record or series           |
| %        | record            | record             | record      | Computes the modulo of the first record's values with the correspondong value in the second record and yields a record with the result, omitting fields and points absent from either record                 |


# Introduction

A general overview of what Service Accounts are and why we use them.

## Overview

All programmatic interaction with our REST API is done using a logged-in Service Account. Once the client has been authenticated, the program interacting with the API will be able to perform operations as an administrator of the organization.

## Service Account vs User Account

There are two types of accounts available; the user and the service account. A user account represents a person, and requires an email and authentication that is suited for user authentication (password or *SSO* using *SAML*). A Service Account represents a program, and the authentication scheme is optimized for machine to machine authentication using `RS256` key pairs or legacy `HS256` shared secrets. Service Account keys can be rotated or configured with an expiration date.

The service account is bound to a single organization, while a user account can be bound to multiple organizations.


# Creating service accounts

A quick guide on how to create a Service Account for our REST API.

## Overview

We will here create a new Service Account using either the web application or our REST API.

## Prerequisites

In order to create the initial service account, the logged in user needs to have a role of Admin.

## Create using app

The first service account needs to be created using our App. Whereas subsequent accounts may be created using the REST API.

1. Navigate to [Service account settings](https://app.neowit.io/settings/serviceAccounts)
2. Click Add
3. Give the account a meaningful name and decide whether you want to enable [Basic Auth](/rest-api/authentication/basic-auth)
4. Click Save
5. Add 1 or more service account keys that may be used for [Authentication](/rest-api/authentication)
6. Choose the key algorithm. Use `RS256` for new integrations; use `HS256` only for legacy shared-secret integrations.
7. Optionally set a key expiration date.
8. Copy the details.

{% hint style="info" %}
For `RS256` keys, store the private key securely; Neowit stores only the public key. For `HS256` keys, the shown secret will not be shown again.
{% endhint %}

{% hint style="danger" %}
Make sure private keys and shared secrets are safely stored. Delete the key if you think it has been compromised in any way.
{% endhint %}

{% hint style="info" %}
If a key has an expiration date, it cannot be used after that date. Create and deploy a replacement key before the current key expires.
{% endhint %}

<figure><img src="/files/zUuTfqK1Qyy6jDsPrF3A" alt=""><figcaption><p>Dialog showing the created key information</p></figcaption></figure>

## Create using REST API

Service accounts may be created, and keys may be rotated or deleted using the REST API shown below. When creating a key through the API, `expiresAt` is an optional Unix timestamp in seconds. See our [API reference](/rest-api/api-reference) for more operations.

{% openapi src="<https://app.neowit.io/api/swagger/doc.json>" path="/service-account/v1/service-account" method="post" %}
<https://app.neowit.io/api/swagger/doc.json>
{% endopenapi %}

{% openapi src="<https://app.neowit.io/api/swagger/doc.json>" path="/service-account/v1/service-account/{id}" method="delete" %}
<https://app.neowit.io/api/swagger/doc.json>
{% endopenapi %}

{% openapi src="<https://app.neowit.io/api/swagger/doc.json>" path="/service-account/v1/service-account/{id}/key" method="post" %}
<https://app.neowit.io/api/swagger/doc.json>
{% endopenapi %}

{% openapi src="<https://app.neowit.io/api/swagger/doc.json>" path="/service-account/v1/service-account/{id}/key/{key-id}" method="delete" %}
<https://app.neowit.io/api/swagger/doc.json>
{% endopenapi %}


# Introduction

Neowit supports a growing list of APIs and ways to get your data into our platform. In addition to using the [REST API ](https://gitlab.com/neowit/docs/-/blob/master/gitbook/integrations/broken-reference/README.md)directly, you can also integrate your edge controllers or applications using [MQTT](https://mqtt.org/).

* [MQTT Integrations](/integrations/mqtt)
  * [Custom integration using Starlark](/integrations/mqtt/custom-starlark)
  * [Native Sparkplug integrations](/integrations/mqtt/native-sparkplug)


# MQTT

Neowit supports MQTT to ingress metrics from your application or edge node controller. Neowit operates an MQTT broker cluster that your application can to connect to.

* [Custom integrations with Starlark transforms](/integrations/mqtt/custom-starlark)
* [Native Sparkplug integrations](/integrations/mqtt/native-sparkplug)

## MQTT broker connection details

| Name               | Description                              |
| ------------------ | ---------------------------------------- |
| Host               | mqtt.neowit.io                           |
| Port               | TCP 8883 - Secure MQTT                   |
| Protocol           | MQTTS (TLSv1.2, TLSv1.3)                 |
| Supported versions | 3.1, 3.1.1, and 5.0                      |
| Username           | Visible during integration configuration |
| Password           | Visible during integration configuration |
| Client ID          | Visible during integration configuration |


# Native Sparkplug

## What is Sparkplug?

Sparkplug is a specification maintained by the Eclipse Sparkplug Working Group under the Eclipse Foundation. Its goal is to provide improved Plug & Play IIoT over MQTT. You can get more information from <https://sparkplug.eclipse.org/>.

Neowit supports the current [Sparkplug 3.0.0](https://sparkplug.eclipse.org/specification/version/3.0/documents/sparkplug-specification-3.0.0.pdf) version of the specification and we provide an MQTT broker and an optional Primary Host Application that you can use to synchronize your Edge Node application with.

## Technical information

### Connecting

The integration settings page will show the connection details you need to use to connect to our MQTT broker. See our [MQTT](/integrations/mqtt) page for more details on this.

### Primary Host Application

Neowit has implemented a Host application that will passively listen to all NBIRTH / NDEATH / DBIRTH / DDEATH / DDATA messages and automatically ingest metrics if they match the criteria described below.

The host will publish its STATE to the following topic: **"spBv1.0/STATE/<:integrationID>"**. If you're app or edge node supports syncing with a Sparkplug Primary Host application, you can set the app id to be id of the integration.

### Supported DBIRTH metric names

If you want the devices and sensors to automatically show up in the Neowit app, you need to specify a set of metrics that Neowit can use to describe the Device and its sensors. This is done in the Sparkplug DBIRTH message, as per the specification the DBIRTH message should contain all metrics that are available on the device. If you want to change the specification after you've sent a DBIRTH you can use NDEATH / NBIRTH as per the specification.

#### Device description metrics

<table><thead><tr><th width="262">Metric Name</th><th>Sparkplug Data Type</th><th>Description</th></tr></thead><tbody><tr><td>/neowit/v1/device/name</td><td>String (12)</td><td>A human readable name</td></tr><tr><td>/neowit/v1/device/vendor</td><td>String (12)</td><td>Vendor of device</td></tr><tr><td>/neowit/v1/device/model</td><td>String (12)</td><td>Model of device</td></tr><tr><td>/neowit/v1/device/deepLink</td><td>String (12)</td><td>Optional http link for more information</td></tr></tbody></table>

#### Sensor description metrics

For each metric you want to be associated with the device, you need to give it a metric name that can be understood by our metric store. The name of the available metric types are available in a table from the integration configuration page. After you found a suitable sensor name, you can construct your metric using the following pattern **"/neowit/v1/sensor/name/:SENSOR\_NAME"**. For example if you want to send a temperature metric, you'd use the metric name **"/neowit/v1/sensor/name/TEMP"**.

#### Sensor Value Data Types

Currently, we support the following Sparkplug data types for ingress into our metrics storage.

| Name    | Data Type |
| ------- | --------- |
| Int8    | 1         |
| Int16   | 2         |
| Int32   | 3         |
| Int64   | 4         |
| UInt8   | 5         |
| UInt16  | 6         |
| UInt32  | 7         |
| UInt64  | 8         |
| Float   | 9         |
| Double  | 10        |
| Boolean | 11        |

### Example DBIRTH Device description

The below DBIRTH (shown as JSON instead of protobuf to make it human readable) will be translated to a Neowit Device as follows:

{% code title="Sparkplug DBIRTH" fullWidth="false" %}

```json
PUBLISH spBv1.0/group1/DBIRTH/node1/device1
{
        "timestamp": 1486144502122,
        "metrics": [{
            "name": "/neowit/v1/device/name",
            "alias": 1,
            "timestamp": 1479123452194,
            "dataType": "String",
            "value": "My Device Name"
        }, {
            "name": "/neowit/v1/device/vendor",
            "alias": 2,
            "timestamp": 1479123452194,
            "dataType": "String",
            "value": "My Trusted Manufacturer"
        }, {
            "name": "/neowit/v1/device/model",
            "alias": 3,
            "timestamp": 1479123452194,
            "dataType": "String",
            "value": "Awesome Device 1.3"
        }, {
            "name": "/neowit/v1/device/deepLink",
            "alias": 4,
            "timestamp": 1479123452194,
            "dataType": "String",
            "value": "https://contoso.com/myDevice/123"
        }, {
            "name": "/neowit/v1/sensor/name/TEMP",
            "alias": 5,
            "timestamp": 1479123452194,
            "dataType": "Double",
            "value": 42.3
        }, {
            "name": "/neowit/v1/diagnostics/name/BATTERY",
            "alias": 6,
            "timestamp": 1479123452194,
            "dataType": "Double",
            "value": 80.3
        }],
        "seq": 2
}
```

{% endcode %}

{% code title="Neowit REST API" %}

```json
{
   "id": "<neowitIdOfDevice>",
   "integrationId": "<neowitMqttIntegrationId>",
   "externalId": "group1.node1.device1",
   "name": "My Device Name",
   "vendor": "My Trusted Manufacturer",
   "deviceName": "Awesome Device 1.3",
   "deepLink": "https://contoso.com/myDevice/123",
   "status": "STATUS_CONNECTED"
   "sensors": ["TEMP"],
   "diagnostics": ["BATTERY"]
}
```

{% endcode %}

### Example DDATA Metrics update

After the Edge application has announced all of its supported sensors in the *DBIRTH* message, it can then start publishing *DDATA* of the update metrics for persisting in our metrics store.

{% code title="Sparkplug DDATA" %}

```json
PUBLISH spBv1.0/group1/DDATA/node1/device1
{
        "timestamp": 1486144503122,
        "metrics": [{

            "name": "/neowit/v1/sensor/name/TEMP",
            "alias": 5,
            "timestamp": 1486144503122,
            "dataType": "Double",
            "value": 30.3
        }, {
            "name": "/neowit/v1/diagnostics/name/BATTERY",
            "alias": 6,
            "timestamp": 1486144503122,
            "dataType": "Double",
            "value": 10
        }],
        "seq": 3
}
```

{% endcode %}


# Custom Starlark

## Introduction

The custom Starlark MQTT integration attempts to make it easy to connect your current MQTT application or Edge controller towards Neowit and write code that transforms the topics and payloads into something that Neowit understands.

## Technical details

### Starlark

The integration uses the Starlark language and our extension modules to the language in order to define devices and metrics that can be stored in the Neowit application. The Starlark language is a simple Python subset that should be easy to learn. It is fairly limited, but should contain enough power to deal with normal JSON MQTT payloads.

See [Starlark documentation](https://gitlab.com/neowit/docs/-/blob/master/gitbook/integrations/mqtt/broken-reference/README.md).

### Connecting to the MQTT broker

The integration settings page will show the connection details you need to use to connect to our MQTT broker. See our [MQTT](/integrations/mqtt) page for more details on this.

### Troubleshooting

The Starlark implementation is intended for advanced users, and we currently don't have any good ways to help you debug the Starlark code.

* If your code does not compile correctly due to syntax errors, the integration will show up as not connected. The built-in editor should give you some hints of what is wrong, but it's not perfect.
* If no devices or metrics show up, you're code may be throwing errors. We currently don't have any good way of showing that to you, but feel free to contact support so that we may help you out.

## Writing code

Our Starlark executor looks for a function called on\_publish with accepts the arguments topic and payload. The topic arguments is the MQTT topic that the payload was published to.

The goal of the code you need to define is to convert the topic and payload into something that Neowit understands using the following modules:

* [devices](/starlark/modules/devices-module)
* [series](/starlark/modules/series-module)
* [sensors](/starlark/modules/sensors-module)

## Examples

### Simple example

Consider the following JSON payload published to topic *mytopic/location1/device1*. The code following would create one unique device and publish a temperature value to the Neowit metrics store.

{% code title="PUBLISH mytopic/location1/device2" %}

```json
{ at: 1719065842, "temperature": 30.3}
```

{% endcode %}

```python
# called when MQTT receives a PUBLISH from the app or edge controller
def on_publish(topic, payload):
    # the payload is JSON formatted, we need to decode
    # it into a starlark data structure.
    data = json.decode(payload)

    # ignore payloads that doesnt have the expected input
    if not "at" in data or not "temperature" in data:
        return
    
    # the topic here includes the information we
    # need to identity a unique device, we use this
    # as our basis. The other attributes are mocked up,
    # but could also be defined if the data is available
    # in this or other payloads.
    external_id = topic.replace("mytopic/", "")
    device = devices.Device(
        external_id=external_id,
        name="My device 1",
        vendor="My device vendor",
        model="My device model",
        status="STATUS_CONNECTED",
        status_reason="Received something"
    )

    # this will register or update the device.
    devices.upsert(device)

    # publish the metric with the given unix timestamp
    # and value on the sensor.TEMP sensor.
    series.publish(external_id, data["at"], sensors.TEMP, data["temperature"])
        
```

### More advanced example

This example decodes a JSON structure and upserts devices and series extracted from the json document.

```python
time_layout = "2006-01-02 15:04:05.000-0700"

##
## Main entry point
##
def on_publish(topic, payload):
  data = json.decode(payload)
  ts = time.parse_time(data["updated"], time_layout)
  if "rooms" in data:
    for room in data["rooms"]:
      publish_room(ts, room)
  if "dampers" in data:
    for damper in data["dampers"]:
      publish_damper(ts, damper)

##
## Upsert damper device and publish series
##
def publish_damper(ts, damper):
  # '564.001-SQ402 Rom 5.1826 .....'
  descr = damper["airflowDescription"].split(" ")
  if len(descr) < 3:
    return

  code, room = descr[0], descr[2]
  id = 'damper%s@%s' % (code, room)
  device = devices.Device(
    external_id = id,
    name = "Damper: %s %s" % (code, room),
    vendor = "My damper vendor",
    model = "My damper model",
    status = "STATUS_CONNECTED",
    status_reason = "OK"
  )
  devices.upsert(device)
  series.publish(id, ts.unix, sensors.AIR_FLOW_CUBIC_HOUR, get_number_or_none(damper, "airflow"))
  series.publish(id, ts.unix, sensors.AIR_FLOW_DEMAND_PERCENTAGE, get_number_or_none(damper, "demandInPrc"))

##
## Upsert room device and publish series
##
def publish_room(ts, room):
  device = devices.Device(
    external_id = room["location"],
    name = "Room: %s" % room["location"],
    vendor = "My vendor",
    model = "My model",
    status = "STATUS_CONNECTED",
    status_reason = "OK"
  )
  devices.upsert(device)
  series.publish(device.external_id, ts.unix, sensors.TEMP, get_number_or_none(room, "temperature"))
  series.publish(device.external_id, ts.unix, sensors.CO2, get_number_or_none(room, "airQuality"))
  series.publish(device.external_id, ts.unix, sensors.MOTION_DETECTED, get_bool_or_none(room, "pir"))
  series.publish(device.external_id, ts.unix, sensors.LIGHT_LUX, get_number_or_none(room, "lightIntensity"))
  series.publish(device.external_id, ts.unix, sensors.AIR_FLOW_DEMAND_PERCENTAGE, get_number_or_none(room, "airDemand"))
  series.publish(device.external_id, ts.unix, sensors.SETPOINT_CO2_PPM, get_number_or_none(room, "setpointCo2Actual"))
  series.publish(device.external_id, ts.unix, sensors.SETPOINT_TEMPERATURE_C, get_number_or_none(room, "setpointActual"))


##
## Helpers
##

def get_number_or_none(room, key):
  if key not in room:
    return None
  value = room[key]
  return None if value == "NULL" else value

def get_bool_or_none(room, key):
  if key not in room:
    return None
  value = room[key]
  return None if value == "NULL" else value

```


# Introduction

## What is Starlark?

Starlark is a small subset of Python developed by Google for their Bazel build system. If you know Python, it should be easy to learn.

### Example code

> The code below is an example of the syntax of Starlark. If you've ever used Python, this should look very familiar. In fact, the code above is also a valid Python code. Still, this short example shows most of the language. Starlark is indeed a very small language. - Starlark README.md

{% code title="From Starlark README.md" %}

```python
# Define a number
number = 18

# Define a dictionary
people = {
    "Alice": 22,
    "Bob": 40,
    "Charlie": 55,
    "Dave": 14,
}

names = ", ".join(people.keys())  # Alice, Bob, Charlie, Dave

# Define a function
def greet(name):
    """Return a greeting."""
    return "Hello {}!".format(name)

greeting = greet(names)

above30 = [name for name, age in people.items() if age >= 30]

print("{} people are above 30.".format(len(above30)))

def fizz_buzz(n):
    """Print Fizz Buzz numbers from 1 to n."""
    for i in range(1, n + 1):
        s = ""
        if i % 3 == 0:
            s += "Fizz"
        if i % 5 == 0:
            s += "Buzz"
        print(s if s else i)

fizz_buzz(20)
```

{% endcode %}

### More Starlark documentation

* [Language specification](https://github.com/bazelbuild/starlark/blob/master/spec.md) (Bazel)
* [Language specification](https://github.com/google/starlark-go/blob/master/doc/spec.md) (starlark-go)

## Modules

We have a few modules available that are available in the global scope. See [Modules](#modules) for more information.


# Modules

Currently, we have the following modules available in the global context

* [time](/starlark/modules/time-module) - Builtin time and duration types and functions
* [json](/starlark/modules/json-module) - Builtin JSON codec
* [math](/starlark/modules/math-module) - Builtin math helper functions
* [devices](/starlark/modules/devices-module) - Functions to define Neowit devices
* [series](/starlark/modules/series-module) - Functions to publish device Neowit metrics
* [sensors](/starlark/modules/sensors-module) - Constants for Neowit sensors


# time module

The original source for this documentation can be found here: <https://github.com/google/starlark-go/blob/master/lib/time/time.go>

Module time is a Starlark module of time-related functions and types.

## Types

The Time and Duration types can be used using the following operators.

<pre class="language-python"><code class="lang-python"><strong>duration + duration = duration
</strong>duration + time = time
duration - duration = duration
duration / duration = float
duration / int = duration
duration / float = duration
duration // duration = int
duration * int = duration
</code></pre>

### Time

A Time represents an instant in time with nanosecond precision. The time type has the following attributes:

#### year: int

The year in which t occurs.

#### month: int

Month returns the month of the year specified by t.

#### day: int

Day returns the day of the month specified by t.

#### hour: int

Hour returns the hour within the day specified by t, in the range \[0, 23].

#### minute: int

Minute returns the minute offset within the hour specified by t, in the range \[0, 59].

#### second: int

Second returns the second offset within the minute specified by t, in the range \[0, 59].

#### nanosecond: int

Nanosecond returns the nanosecond offset within the second specified by t, in the range \[0, 999999999].

#### unix: int

Unix returns t as a Unix time, the number of seconds elapsed since January 1, 1970 UTC. The result does not depend on the location associated with t. Unix-like operating systems often record time as a 32-bit count of seconds, but since the method here returns a 64-bit value it is valid for billions of years into the past or future.

#### unix\_nano: int

Returns t as a Unix time, the number of nanoseconds elapsed since January 1, 1970 UTC. The result is undefined if the Unix time in nanoseconds cannot be represented by an int64 (a date before the year 1678 or after 2262). Note that this means the result of calling UnixNano on the zero Time is undefined. The result does not depend on the location associated with t.

### Duration

A Duration represents the elapsed time between two instants as an int64 nanosecond count. The representation limits the largest representable duration to approximately 290 years.

#### hours: float

Hours returns the duration as a floating point number of hours.

#### minutes: float

Minutes returns the duration as a floating point number of minutes.

#### seconds: float

Seconds returns the duration as a floating point number of seconds.

#### milliseconds: int64

Milliseconds returns the duration as an integer millisecond count.

#### microseconds: int64

Microseconds returns the duration as an integer microsecond count.

#### nanoseconds: int64

Nanoseconds returns the duration as an integer nanosecond count.

## Constants

The module defines the following constants:

* **nanosecond** - A Duration representing one nanosecond.
* **microsecond** - A Duration representing one microsecond.
* **millisecond** - A duration representing one millisecond.
* **second** - A Duration representing one second.
* **minute** - A Duration representing one minute.
* **hour** - A Duration representing one hour.

## Functions

The module defines the following functions.

### def from\_timestamp(sec: int, nsec: int) -> Time:

Converts the given Unix time corresponding to the number of seconds and (optionally) nanoseconds since January 1, 1970 UTC into an object of type Time. For more details, refer to <https://pkg.go.dev/time#Unix>.

### def is\_valid\_timezone(loc: str) -> bool:

Reports whether loc is a valid time zone name.

### def now() -> Time:

Returns the current local time.

### def parse\_duration(d: str) -> Duration:

Parses the given duration string. For more details, refer to <https://pkg.go.dev/time#ParseDuration>.

### def parse\_time(x: str, format: str = None, location: str = None) -> Time:

Parses the given time string using a specific time format and location. The expected arguments are a time string (mandatory), a time format (optional, set to RFC3339 by default, e.g. "2021-03-22T23:20:50.52Z") and a name of location (optional, set to UTC by default). For more details, refer to <https://pkg.go.dev/time#Parse> and <https://pkg.go.dev/time#ParseInLocation>.

### def time(year: int, month: int, day: int, hour: int, minute: int, second: int, nanosecond: int, location: str) -> Time:

Returns the Time corresponding to yyyy-mm-dd hh:mm:ss + nsec nanoseconds in the appropriate zone for that time in the given location. All the parameters are optional.


# json module

The original source for this documentation can be found here: <https://github.com/google/starlark-go/blob/master/lib/json/json.go>

Module json is a Starlark module of JSON-related functions.

## def encode(x):

The encode function accepts one required positional argument, which it converts to JSON by cases:

* A Starlark value that implements Go's standard json.Marshal // interface defines its own JSON encoding.
* None, True, and False are converted to null, true, and false, respectively.
* Starlark int values, no matter how large, are encoded as decimal integers. Some decoders may not be able to decode very large integers.
* Starlark float values are encoded using decimal point notation, even if the value is an integer. It is an error to encode a non-finite floating-point value.
* Starlark strings are encoded as JSON strings, using UTF-16 escapes.
* A Starlark IterableMapping (e.g. dict) is encoded as a JSON object. It is an error if any key is not a string.
* Any other Starlark Iterable (e.g. list, tuple) is encoded as a JSON array.
* A Starlark HasAttrs (e.g. struct) is encoded as a JSON object.

If an application-defined type matches more than one the cases describe above, (e.g. it implements both Iterable and HasFields), the first case takes precedence. Encoding any other value yields an error.

## def decode(x\[, default]):

The decode function has one required positional parameter, a JSON string. It returns the Starlark value that the string denotes.

* Numbers are parsed as int or float, depending on whether they contain a decimal point.
* JSON objects are parsed as new unfrozen Starlark dicts.
* JSON arrays are parsed as new unfrozen Starlark lists.

If x is not a valid JSON string, the behavior depends on the "default" parameter: if present, decode returns its value; otherwise, decode fails.

## def indent(str, \*, prefix="", indent="\t"):

The indent function pretty-prints a valid JSON encoding, and returns a string containing the indented form. It accepts one required positional parameter, the JSON string, and two optional keyword-only string parameters, prefix and indent, that specify a prefix of each new line, and the unit of indentation.


# math module

The original source for this documentation can be found here: <https://github.com/google/starlark-go/blob/master/lib/math/math.go>

Module math is a Starlark module of math-related functions and constants. All functions accept both int and float values as arguments. The module defines the following constants and functions.

## e

The base of natural logarithms, approximately 2.71828.

## pi

The ratio of a circle's circumference to its diameter, approximately 3.14159.

## def ceil(x):

Returns the ceiling of x, the smallest integer greater than or equal to x.

## def copysign(x, y):

Returns a value with the magnitude of x and the sign of y.

## def fabs(x):

Returns the absolute value of x as float.

## def floor(x):

Returns the floor of x, the largest integer less than or equal to x.

## def mod(x, y):

Returns the floating-point remainder of x/y. The magnitude of the result is less than y and its sign agrees with that of x.

## def pow(x, y):

Returns x\*\*y, the base-x exponential of y.

## def remainder(x, y):

Returns the IEEE 754 floating-point remainder of x/y.

## def round(x):

Returns the nearest integer, rounding half away from zero.

## def exp(x):

Returns e raised to the power x, where e = 2.718281… is the base of natural logarithms.

## def sqrt(x):

Returns the square root of x.

#### def cos(x):

Returns the arc cosine of x, in radians.

## def asin(x):

Returns the arc sine of x, in radians.

## def atan(x):

Returns the arc tangent of x, in radians.

## def atan2(y, x):

Returns atan(y / x), in radians. The result is between -pi and pi. The vector in the plane from the origin to point (x, y) makes this angle with the positive X axis. The point of atan2() is that the signs of both inputs are known to it, so it can compute the correct // quadrant for the angle. For example, atan(1) and atan2(1, 1) are both pi/4, but atan2(-1, -1) is -3p&#x69;*/*&#x34;.

## def cos(x):

Returns the cosine of x, in radians.

## def hypot(x, y):

Returns the Euclidean norm, sqrt(*x*x + y\*y). This is the length of the vector from the origin to point (x, y).

## def sin(x):

Returns the sine of x, in radians.

## def tan(x):

Returns the tangent of x, in radians.

## def degrees(x):

Converts angle x from radians to degrees.

## def radians(x):

Converts angle x from degrees to radians.

## def acosh(x):

Returns the inverse hyperbolic cosine of x.

## def asinh(x)

Returns the inverse hyperbolic sine of x.

## def atanh(x):

Returns the inverse hyperbolic tangent of x.

## def cosh(x):

Returns the hyperbolic cosine of x.

## def sinh(x):

Returns the hyperbolic sine of x.

## def tanh(x):

Returns the hyperbolic tangent of x.

## def log(x, base):

Returns the logarithm of x in the given base, or natural logarithm by default.

## def gamma(x):

Returns the Gamma function of x.


# devices module

## Introduction

The device module is a collection of types and functions that enables the integration to defined Neowit devices in the database.

### Example

This snippet defines a new Device and registers it to the database.

```python
device = devices.Device(
  external_id = "myuniquedeviceid1",
  name = "Temperature room 1",
  vendor = "My sensor vendor",
  model = "My sensor model",
  status = "STATUS_CONNECTED",
  status_reason = "OK"
)
devices.upsert(device)
```

## Types

### Device

A Device represents a Neowit Device. It needs an identifier that is unique among the devices for the specific integration.

#### external\_id: str

The integration unique id of the device.

## Functions

### def Device(external\_id: str, name: str = None, status: str = None, status\_reason: str = None, vendor: str = None, model: str = None) -> Device:

Creates a new Device. External id is the only required attribute, but we recommend that you provide as much information as possible. Status can be of the following values:

* **STATUS\_CONNECTED**: The device is connected.
* **STATUS\_NOT\_CONNECTED**: The device is not connected.
* **STATUS\_UNKNOWN**: The status of the device is not known.

### def upsert(device: Device):

Will register or update the device with the provided attributes. First time this is called, a new device with external\_id will be added to the database. After this the attributes will be updated if there are any changes.


# series module

## Introduction

This modules enables publishing metrics towards the Neowit metrics store.

## Example

This example defines and registers a new [devices.Device](/starlark/modules/devices-module) and then publishes a temperature sensor metric towards Neowit metrics store.

```python
# defined the device and its attributes
device = devices.Device(
  external_id = id,
  name = "My temperature sensor in room 302",
  vendor = "Contoso Galactic",
  model = "HotHot",
  status = "STATUS_CONNECTED",
  status_reason = "OK"
)
# register or update the attributes of the device
devices.upsert(device)

# publish the new temperature metric
series.publish(device.external_id, time.now().unix, sensors.TEMP, 30.2)
```

## Functions

### def publish(external\_id: str, timestamp: int64, sensor: [SensorType](/starlark/modules/sensors-module), value: number = None):

Publishes a new metric value for the device with external\_id that occured on the current time of execution with a sensor type of [sensors.TEMP](/starlark/modules/sensors-module)


# sensors module

## Introduction

The sensors modules provides constants for the different sensors Neowit supports. A full table of values is provided on the integration configuration page.

## Example

```python
# Air Flor per cubic hour
sensorType = sensors.AIR_FLOW_CUBIC_HOUR

# Indoor temperature in Celsius
sensorType = sensors.TEMP

# Outdoor temperature in Celsius
sensorType = sensors.OUTDOOR_TEMP
```


# Introduction


# Create users using API

A guide on how to create users with our API

## Overview

This guide aims to provide the basic understanding and ability to add users to the Neowit platform using the API.

API Reference: <https://app.neowit.io/api/swagger/index.html#/user/post_user_v1_user>

## Prerequistes

To use this API you must have an access token to a principal that is an organization admin (see [REST API/Authentication](/rest-api/authentication/oauth2)).

## Example

The example code in this page is provided as is, it may not work in your environment and should be used as a quick guide for implementation rather than code to be used in a production environment.

### Environment Setup

The following packages are required by the example code and must be installed.

```bash
pip install requests
```

### Source Code

If you wish to run the code locally, make sure you have a working runtime environment.

```python
import requests # pip install requests

user_endpoint = 'https://app.neowit.io/api/user/v1/user'
token='' # add token here

def create_user(access_token):
    data = {
        'email':    'testuser321@contoso.com',
        'name':     'testuser321',
        'locale':   'en-US',
        'role':     'ROLE_MEMBER', # ROLE_MEMBER or ROLE_ADMIN
        'timezone': 'Europe/Oslo',
        'idpId':    '', # blank if username/password
    }

    return requests.post(
        url=user_endpoint,
        headers={
            'Authorization': 'Bearer ' + access_token,
            'Content-Type':  'application/json',
        },
        json=data,
    )
    
def main():
    print(create_user(token).json())

if __name__ == '__main__':
    main()
```

### Customizing user authentication

If you wish to add a specific Idp you can go to <https://app.neowit.io/settings/idps> then select your Idp and get the ID from the URL, it will have the following format <https://app.neowit.io/settings/idps/edit/>*\<ID>.*