README.md 21 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521
# Arena Web Hub for the Web of Things

Arena Web Hub is an open source implementation of a Web of Things application hub (i.e. a Web Hub) as a node module released under the MIT license to encourage widespread adoption. It integrates a Web server for access by Web browsers.

* If you have any questions please contact the maintainer: Dave Raggett <dsr@w3.org>

## Installation

The easiest way to install the Arena Web Hub is with npm using the command line:

```
npm arena-webhub
```

If you are interested in contrubuting to further development, including providing example applications and library modules for IoT technologies, you are encouraged to clone the GitHub project at <https://github.com/draggett/arena-webhub>.  If you have a GitHub account, you are also welcome to add to the issue tracker. See below for information on using the Web Hub.

Arena has minimal external dependencies and is compact with under 2000 lines of JavaScript. It makes use of the [jshashes node module](https://github.com/h2non/jshashes) for computing the SHA-1 based handshake used for upgrading from HTTPS to WSS. Applications are likely to have dependencies on other modules, e.g. for managing JSON Web Tokens, and for utilising IoT technologies like Bluetooth and ZigBee.

### Server certificates

The server looks for "cert.pem" and "key.pem" in the directory it is run in.  You can create self-signed certificates for "localhost" using openssl, e.g.

```
openssl req -newkey rsa:2048 -x509 -nodes -keyout key.pem -new -out cert.pem -subj /CN=localhost -reqexts SAN -extensions SAN -config <(cat /System/Library/OpenSSL/openssl.cnf \
    <(printf '[SAN]\nsubjectAltName=DNS:localhost')) -sha256 -days 3650
```

Note that browsers will warn users that a secure connection cannot be made.

On Chrome you can inform the browser that you trust this certificate.

On Safari you will need to follow these steps:

1.	Locate where your certificate file is. It is likely to be
  somewhere near your web server configurations.
2.	Open up Keychain Access. You can get to it from
  Application/Utilities/Keychain Access.app.
3.	Drag your certificate into Keychain Access.
4.	Go into the Certificates section and locate the certificate you just added
5.	Double click on it, enter the trust section and under
  “When using this certificate” select “Always Trust”

## Future plans

* This module focuses on enabling server-side applications to expose things for use by clients. A complementary module is planned for enabling client-side applications to access things exposed by other clients. The challenge is how to support different Web Hubs given the current lack of interoperability due to variations in how they make use of protocols like HTTP and WebSockets.
* Another scenario is where there are a set of Web Hubs that are all behind different firewalls. The challenge is how to arrange for a peer to peer virtual network that spans these firewalls in a secure manner. W3C's WebRTC standard could be useful as it supports peer to peer data transfer and is supported by Web browsers.

## Security and user account management

Arena supports HTTPS, Server-Sent Events and Web Sockets (WSS) over the same IP port with transport layer encryption and JSON Web Tokens (JWT) for authorisation. Applications are responsible for managing user accounts, generating, validating and revoking JSON Web Tokens. To support account management, applications are responsible for handling HTTPS requests under a configurable top level path (the default is "/account"). Validation of JSON Web Tokens must be implemented via an application supplied call back.

Applications should provide a means for users to register accounts, to login or out, to review and update their account details, and to unregister as needed. JSON Web Tokens may be created when the user logs in and revoked when the user explicitly logs out or unregisters. Applications are recommended to use second factors as part of the process for activating new accounts. An email address can be sent a message with a time limited link that the user clicks on to activate the account. Alternatively, a mobile phone number may be sent an SMS with a randomly generated time limited numeric code that the user enter into the login page to activate the account.

The email address and mobile number may also be used by applications to send users notifications as appropriate, however this require the user's explicit approval and should be updateable as part of the user's account details. Applications are further required to conform with applicable data privacy regulations, such as Europe's GDPR, in respect to data handling policies.

## IoT technologies

Applications may use any suitable technologies for accessing IoT devices, e.g. Bluetooth and ZigBee, for which there are existing node modules. A similar situation applies to accessing devices belonging to particular ecosystems, e.g. Apple HomeKit, Android Things, the Open Connectivity Foundation (OCF) and oneM2M. A major benefit of the Web of Things is to shield client applications from the IoT technologies at the network edge.

## Web of Things

The IoT is fragmented by incompatible platforms, and myriad technologies and standards. This creates market friction through raising costs and risks. W3C's Web of Things addresses this by introducing an abstraction layer with things that stand for sensors, actuators and associated information services. 

* Server applications expose things, hiding the details of the IoT technologies and standards
* Client applications interact with things as software objects with properties, actions and events
* Things are identified with URLs that can be dereferenced to download human readable descriptions as HTML, or machine interpretable descriptions as JSON-LD
* RDF and Linked Data can be used for rich descriptions of the kinds of things, their capabilities, interrelationships, and the context in which they reside. As an example, a smart home could be described in terms of the different kinds of smart lights in each room.

![WebHubs](www/webhubs.png)

Web Hubs are an important new class of Web application platforms that enable open markets of services where suppliers and consumers of services are connected via the Web of Things using Web protocols for easy integration with Web browsers and other Web Hubs.

## Usage

You are encouraged to look at the examples folder for example applications.

```javascript
let webhub = require('arena-webhub')({
	port: 8888,
	accountPath: '/account',
	accountManager: function (request, response) {
        // handle HTTPS client
    },
    validateJWT: function (jwt, path) {
        // verify JSON Web Token returning true or false
    }
});
```

The port and account path are optional and default to 8888 and "/account". The accountManager and validateJWT are required application callbacks. The accountManager callback is passed two arguments for the HTTPS request and response (see the HTTPS node module), and is responsible for handling the request and the associated response. The validateJWT callback is passed two arguments for the JSON Web Token and the URL path for the connection. The return value is true if the token is currently valid for this path, or false if it isn't.

The Arena Web Hub module exports a single function for applications to expose things for access by clients. Produce is passed a thing description as a JSON object and returns a promise for the exposed thing, for example:

```
webhub.produce({
    name: "light12",
    description: "hall light"
    properties: {
        brightness: {
            type: "integer",
            minimum: 0,
            maximum: 100
        }
    },
    actions: {
    },
    events: {   
    }
}).then (thing => {
    // initialise thing's behavior
}).catch (err => {
    // handle error
})
```

## Integrated Web Server

Web applications for access from Web browsers involve a set of  Web page resources. The Arena Web Hub looks for these in the "./www" folder. The integrated Web server will serve up "./www/index.html" for URL GET requests with the path "/".  Applications are responsible for managing Web resources  for the Web server. This includes web page scripts and their use of cookies.

## Thing Descriptions

The object interface for things is described using JSON-LD with a default context. Arena Web Hub follows the W3C Thing Description specification, see: https://www.w3.org/TR/wot-thing-description/

Note that the specification is still evolving, and the Arena Web Hub may not be fully compliant to the latest version of the [editor's draft specification](https://w3c.github.io/wot-thing-description/). An example is that Arena doesn't currently support protocol binding templates using "forms". This is because protocol binding templates aren't yet capable of a complete description of Arena's use of Server-Sent Events and WebSockets.

## Thing API

This section describes the API exposed by the Arena Web Hub for server-side applications. 

Each thing exposes the following interface:

```json
class Thing {
    id // string - a unique id for the thing
    name // string - a human friendly name for the  thing
    model // JSON object for the thing description
    properties // object - a map from name to property
	actions // object - a map from name to action
	events // object - a map from name to event
	addActionHandler(name, callback) // set callback for named action
}
```

Properties expose the following interface:

```json
class ThingProperty {
    thing // the thing this property belongs to
	name // the name of this property
    value // the current value of this property
	write(value) // to update the current value and notify clients
    subscribe(observer) // add observer to list of observers
    unsubscribe(observer) // remove observer from list of observers
}

function observer (data) {
    // notification of updated value for property
}
```

Actions expose the following interface, where optional timeout is in milliseconds, and the handler to be called is set using addActionHandler, see above, where the callback takes a single argument with the input for the action, and returns a promise that resolves to the output from the action.

```json
class ThingAction {
    thing // the thing this property belongs to
	name // the name of this property
	invoke(input [,timeout]) // invoke with input, returns promise for the output
}
```

Events expose the following interface:

```json
class ThingEvent {
	thing // the thing this property belongs to
	name // the name of this event
	emit(data) // to notify clients of this event with given data
    subscribe(observer) // add observer to list of observers
    unsubscribe(observer) // remove observer from list of observers
}
```

### Remote Access

A common situation is where the hub that is exposing a thing is behind a firewall and wants to make it available to applications outside of the firewall, which are blocked by the firewall from connecting as regular clients. The Arena Web Hub can be used in the following ways:

* As the hub behind the firewall with an application that seeks to expose a thing remotely

  * This application needs to get authorisation for the remote access from the external hub which should return the URI for a Web Socket (WSS) connection along with the JASON Web Token (JWT)  for use when establishing that connection.

  * The application can then ask the hub to open and add the connection for the given thing.

    ``` javascript
    thing.addRemoteClient(uri, jwt)  // returns promise for when established
    ```

* As a hub outside of the firewall that republishes (i.e. proxies) a thing from behind the firewall

  * The application on this hub needs to authorise the proxying of things from behind the firewall using the URL space it reserved when configuring the hub.

  * The application on this hub needs to authorise access for things from behind the firewall using the URL space it reserved when configuring the hub.

    ```javascript
    thing.proxy(jwt);  // republish the thing that connects using this token
    ```

* As a hub outside of the firewall that acts as the sole client for a thing exposed behind the firewall, for example, where a device vendor seeks to monitor the operation of a device in order to provide predictive maintenance.

  * The application needs to designate the JSON Web Token that will be used by the remote hub that owns the thing

    ```javascript
    thing.setOwner(jwt); // provide access to the thing that uses this token
    ```

## Protocols

This section describes how the Arena Web Hub makes use of HTTPS, Server-Sent Events and WebSockets. In practice, this is implemented as a client-side library that hides the protocols behind a simple API. See "wot.js" in the examples folder for a library that also supports Siemens' ThingWeb and Mozilla's Things Gateway platforms. Note that the client side API needs a means for applications to subscribe and unsubscribe to events as this is required for some protocols but not others.

Clients access thing descriptions with an HTTPS GET request with  JWT for authorisation.

```javascript
const uri = "https://localhost:8888/things/light12";
const opts = {
	method: "GET",
	headers: {
		'Authorization': `Bearer ${jwt}`,
		'Accept': 'application/json'
	}
};

fetch(uri, opts).then(response => {
	if (response.ok) {
		resolve(response.json());
	}
	else
		throw(new Error("response status = " + response.status));
}).catch(err => {
	console.log("couldn't get thing description at " + uri);
	reject(err);
});
```

Clients can request the set of currently exposed things with a GET to "/things" which returns a JSON array where the items are the thing descriptions for each exposed thing.  This requires JWT for authorisation.

Clients may use GET for a given property, e.g.

```javascript
const uri = "https://localhost:8888/things/light12/properties/brightness";
const opts = {
	method: "GET",
	headers: {
		'Authorization': `Bearer ${jwt}`,
		'Accept': 'application/json'
	}
};

fetch(uri, opts).then(response => {
	if (response.ok) {
		resolve(response.json());
	}
	else
		throw(new Error("response status = " + response.status));
}).catch(err => {
	console.log("couldn't get thing description at " + uri);
	reject(err);
});
```

Clients may also request the current state for all the properties as a JSON object:

```javascript
const uri = "https://localhost:8888/things/light12/properties";
const opts = {
	method: "GET",
	headers: {
		'Authorization': `Bearer ${jwt}`,
		'Accept': 'application/json'
	}
};

fetch(uri, opts).then(response => {
	if (response.ok) {
		resolve(response.json());
	}
	else
		throw(new Error("response status = " + response.status));
}).catch(err => {
	console.log("couldn't get thing description at " + uri);
	reject(err);
});
```

To update the value of a property, clients can use a PUT on the path for the property with the new value encoded as JSON as the body of the request, e.g.

```javascript
const uri = "https://localhost:8888/things/light12/properties/brightness";
console.log('uri: ' + uri);

const opts = {
	method: "PUT",
	headers: {
        'Authorization': `Bearer ${jwt}`,
		'Accept': 'application/json',
		'Content-Type': 'application/json'
	},
	body: JSON.stringify(value)
};

fetch(uri, opts).then(response => {
	if (response.ok)
		resolve(true);
	else
		throw(new Error("property is unknown or unwritable"));
}).catch(err => {
	console.log("couldn't set property to " + JSON.stringify(value));
	reject(err);
});
```

Clients can update the value for multiple properties with a PUT to the path "/properties" with body as a JSON object with the corresponding properties and their new values.

To invoke an actions clients may send an HTTP POST request to the path for the action with the body as the JSON representation of the input data to be passed to the action. The response body contains the JSON representation of the output of the action. For example:

```javascript
const uri = "https://localhost:8888/things/light12/actions/dim";
console.log('uri: ' + uri);

const opts = {
	method: "POST",
	headers: {
        'Authorization': `Bearer ${jwt}`,
		'Accept': 'application/json',
		'Content-Type': 'application/json'
	},
	body: JSON.stringify(input)
};

fetch(uri, opts).then(response => {
	if (response.ok)
		resolve(response.json());
	else
		throw(new Error("action is unknown or disallowed"));
}).catch(err => {
	console.log("couldn't invoke action with " + JSON.stringify(input));
	reject(err);
});
```

Note that Arena will reject data that doesn't match the constraints in the thing description whether this comes from the server or from client applications.

Clients may listen for all events using server-sent events on the path "/events". The stream sends either a regular event as declared in the thing description, or a notification of a new value for a property, or for a set of such properties, e.g.

```javascript
let eventSource = new
	EventSource(`https://localhost:8888/things/light12/events?jwt=${jwt}`);

eventSource.onmessage = function(e) {
	console.log("received event: " + e.data);
	let data = JSON.parse(e.data);
	
	if (data.event) {
		let eventName = data.event;
		let eventValue = data.data;
	} else if (data.property) {
        let propertyName = data.property;
        let propertyValue = data.data;
	} else if (data.state) {
        // data.state is an object mapping property names to values
	}
}
```

Alternatively clients can use HTTPS GET in long polling mode on a particular event, or property, e.g.

```javascript
const uri = thing_uri + '/events/' + name;

var poll = function () {
	const opts = {
		method: "GET",
		headers: {
			'Accept': 'application/json'
		}
	};

	fetch(uri)
		.then(res => res.json())
		.then(value => {
			//console.log(name + ': ' + JSON.stringify(value));
			if (handler) {
				handler(value);
			}
			setTimeout(poll, 0); // to wait for next event
		}).catch(err => {
			console.log("couldn't get event from " + uri);
		});
};

setTimeout(poll, 0);  // kick off polling
```

### Using Web Sockets

This version of the Arena Web Hub assumes a single Web Socket connection per thing. In future, you will be able to share the connection by opening the Web Socket server on the "/things" path rather than on the path for a specific thing.

Here's an example of  how to open a Web Socket for a particular thing.

```javascript
let thing_uri = "https://localhost:8888/things/light12";
let ws = new WebSocket(`${thing_uri}?jwt=${jwt}`);
    	
console.log("websocket connection opening ...");

ws.onopen = function() {
	console.log("websocket connection opened");
};

ws.onclose = function() {
	console.log("websocket connection closed");
};

ws.onerror = function() {
	console.log("websocket connection error");
};

ws.onmessage = function(message) {
	console.log("received message: " + message.data);
	try {
		let data = JSON.parse(e.data);

		if (data.event) {
			let eventName = data.event;
			let eventValue = data.data;
            // do something with event
		} else if (data.property) {
        	let propertyName = data.property;
        	let propertyValue = data.data;
            // do something with property update
		} else if (data.state) {
        	// data.state is an object mapping property names to values
        } else if (data.status) {
            // a response to a client request
            let id = data.id;  // identifies client request
            let status = data.status; // same codes as for HTTP
            // match response to request and do something 
        }
        
	} catch (err) {
		console.log('badly formed message: ' + message.data)
	}
};
```

Clients can push property updates individually or together:

```javascript
let message = {
	id: 23,
    property: "brightness",
    data: 42
};
ws.send(JSON.stringify(message));

message = {
	id: 24,
    state: {
        property1: value1,
        property2: value2,
        ...
    }
};
ws.send(JSON.stringify(message));
```

The id should uniquely identify each request and is used to match the request to the response sent by the server. A simple approach is to associate a counter with the connection and increment for each request. Care is needed when re-opening a socket as the server may send responses for requests received with the previous connection.

The server response looks like the following:

```json
// a successul transaction
{
    id: 23,
    status: 200
}

// a failed transaction (invalid data)
{
    id: 23,
    status: 400
    description: "invalid data"
}
```

The status codes are the same as for the HTTP protocol. The description field is optional.

Clients can invoke actions and handle the responses as follows:

```json
// invoke action'start' with input data 13
{
    id: 71,
    action: "start",
    input: 13
}

// a successful action response with output true looks like
{
    id: 71,
    status: 200,
    output: true
}

// a failed action looks like
{
    id: 71,
    status: 500,
    description: "server error"
}
```

Arena will check that the input and output match the type declarations in the thing description. The action will fail if either of these are invalid, or if the action name is not declared in the thing description.  If the action handler itself fails, it can either reject the promise or it can resolve the promise and use output to describe the failure.  Note that if the server application code throws an exception, this will result in a failed action as above.