Author Archives: Dr.Jukka

Graph control for Windows 10 UWP platform

The standard UWP XAML controls are covering most of the use case rather nicely, but there are gaps on support when it comes to visualization of data. For example there are no readily available controls showing data as a graph.

One of customers then needed a simple demo application, in which they wanted to show dome data with line graph, so I went to designing one for them.  To develop and test this control, I decided to create an app that could read heartrate sensor and then visualize the data in the screen.

I did find a very good starting point for this exercise from Microsoft examples and downloaded the Bluetooth Generic Attribute Profile – Heart Rate Service  example from code.msdn.microsoft.com

I did took the BLE service parts from it and did modify them to suit my requirements better, but the main thing from there was the ChartControl it implemented. It’s basically a very simple control that draws the heart rate sensor data into a UI, it is not implementing any additional functionality such as zooming etc.

All and all, it made a great starting point. I copied the file over to a new project, and started adding zooming into it. I also did quite a lot of re-structuring as well as changed how the background was drawn. While keeping most of the architecture intact I did add loads of constants, so it would be easier to customize the control for different usages.  You can find the final version in ChartControl.cs  file in GitHub.

Then I needed to add zooming into the control. I also wanted to have a way on showing the current heart rate as well as a way user could save the data gathered into a file. These generally would require some UI changes, and I also wanted to keep the original ChartControl control in its original form, just to be able to offer a very basic control for those users who would need one.

So I made another control I named ChartControlFull. This control has XAML part as well, so defining UI’s was just normal XAML stuff. I designed the data saving to be handled in a way that the page to which  the control is added needs to implement delegate to be called once user presses the save button. Then in the delegate, the actual data can be fetched using getDataString() function. The function return comma separated string where the data is in ushort format. The data returned is always just the visible area in the control, so users can use zooming to select how much data will be stored. The actual file etc. storing need to be handled by the calling page’s logic. You can find the final versions for ChartControlFull.xaml, and ChartControlFull.xaml.cs files in GitHub. Video below shows how this control looks & behaves.

This control works fine when we are just showing the incoming Heart rate data, since in general the data amount is rather small and the rate the data is coming in is rather slow. I did also add 0.3 second delay on re-draw after zoom button presses, and if zoom is changed within the time, the previous issued re-drawing is cancelled, this is general makes the zooming effect to work faster, since all rapid changes are buffered before executing the slow full re-draw.

The slowness comes from the fact that the control will always re-draw all parts when anything changes, i.e. whether we change the zoom, or we get new data. This means that if we have loads of data (I tested with around 15’000 data points), it becomes really slow. Also while testing it, I noticed that I’m not getting the pointer events delivered consistently, making it rather impossible to implement any reliable gesture actions.

First though I had for fixing this was to try drawing the graph into a bigger sized canvas object, and then rendering the canvas into an image utilizing RenderTargetBitmap API. The API is great and simple to use, but as documented in XAML visuals and RenderTargetBitmap capture capabilities  it can only render visible UI components, thus it would not work on my use case.

Luckily I did find an API which is 100% match for my requirements, and it’s available through NuGet. This API is called Win2D, and it allows creating off-screen canvas object, which can be drawn into the UI when needed. Also it does allows creating multiple objects and then combining them in the drawing function.

I named the new control as ChartWin2DControl, and you can find the implementation from ChartWin2DControl.xaml, and ChartWin2DControl.xaml.cs files in GitHub. Video below shows how this control looks & behaves.

The basic drawing implementation is still the same as was used with the original ChartControl, but with this implementation I’m using 2 off-screen canvas objects. The first object is having same size as the control has. This is required since it will be hosting the background lines as well as the value texts, which should not be scaled.

The second off-screen canvas then has transparent background, and only thing that gets drawn to it, is the actual line Graph data. Since the control is not implementing any vertical zoom effects, the height of the canvas is same as with the control. The optimal value for the width in pixels then would be at least the amount of data points we have. This would allow the drawing code to move the x-axis to the right with each data item.

Unfortunately different hardware’s are having different maximum width/height restrictions, thus the actual width of the canvas is determined by the value returned by CanvasDevice.MaximumBitmapSizeInPixels Property.

For gestures, as this control should work also with UWP apps run with devices not having touch screens, I decided that all gestures should work with single pointing device, and thus I’m not utilizing the actual gestures API in the control. Instead I opted on just implementing pointer move detection with PointerMoved event handler.

The needed action for the app was zoom in/out, and then while being zoomed in, we also need to be able to move the graph image horizontally.

The implementation is rather simple, in it each action is always started with PointerPressed event, which after there is separate threshold values defined for vertical and horizontal movements to determine which action should be performed. And once the movement is more than defined in the threshold value, then the action is started. If it was the horizontal threshold that was reached first, then all PointerMoved events would be used for moving the zoomed graph horizontally.

And if the vertical threshold was reached first, then all following PointerMoved events would be used for zooming the graph.

Any PointerReleased, PointerCanceled, PointerCaptureLost, PointerExited would end the action, and new action would then be started only after we get new PointerPressed event.

The actual zooming effect is then implemented by changing the drawing source rectangle inside the graph canvas. And as we are not implementing any vertical zooming, the zoom action will only change the width of the rectangle size.

And then once we get moving actions, it will simple reposition the source rectangle on the graph canvas.

Full implementation available at GitHub: DrJukka/Heart-rate-monitor-UWP-, also you can find the application from Microsoft Store : Heartbeat monitor

Using Bluetooth with Windows 10

I went through Using BLE GATT services with Windows 10 in previous post, and in this I’m going to go through the codes used for creating serial communication through the Bluetooth connection.

Full examples for the use cases can be found under BLETestStuffWindows section under my GitHub repos. An application handling the communications on foreground is called BluetoothApp, and application implementing background Bluetooth listener is called BluetoothAppBG.

In connecting device side the code is pretty much as it is with any other service with Windows 10, i.e. you need to use the DeviceInformation.FindAllAsync() to find the device you want to connect to. And in order to get your destination device into the returned list, you do need to pair it beforehand, thus do remember to do that before you start connection process, though do remember to start advertising for the service before pairing, otherwise its not included in the cached pairing information.

The filter string is created with  RfcommDeviceService GetDeviceSelector function with your own GUID:

Guid btUuid = Guid.Parse("6f5b75e8-be27-4684-a776-3238826d1a91");
string deviceFilter = RfcommDeviceService.GetDeviceSelector(RfcommServiceId.FromUuid(btUuid)); 
var devices = await DeviceInformation.FindAllAsync(deviceFilter); 

The devices array then includes all paired devices which did implement a listener to the specified GUID in the time of pairing the device. Then to fetch bluetooth related information and to be able to connect to the device, you need to  RfcommDeviceService instance for a selected device.

RfcommDeviceService service = await RfcommDeviceService.FromIdAsync(selectedDevice.Id);

 

Then in case the selected device did add SDP attributes for advertising, you could actually fetch them here, even before you do any connection to the device.

var attributes = await service.GetSdpRawAttributesAsync(BluetoothCacheMode.Uncached);

You could use the DataReader to read the actual attributes. Do note that if the service uses static SDP attributes, then you don’t need to use the BluetoothCacheMode.Uncached attribute. In this case you would get the attributes read from the device, while it was paired.

Using BluetoothCacheMode.Uncached attribute allows the service to update the attributes when needed, this could be used for example to tell any clients the last update on any data the service is proving, thus devices who have already connected after the last update, would not need to do actual connection to the service to find that out.

Then the actual connection to the remote device is handled with StreamSocket, and once the connection is established, the StreamSocket  instance is used for reading & writing data between the devices.

StreamSocket streamSocket = new StreamSocket();
await streamSocket.ConnectAsync(
 service.ConnectionHostName,
 service.ConnectionServiceName,
 SocketProtectionLevel.BluetoothEncryptionAllowNullAuthentication);

On the receiving side, you can use RfcommServiceProvider to make the Bluetooth service visible to other devices, and then use StreamSocketListener to listen for incoming connections. The usage of these classes is pretty strait forward, you simply create a provider with your own GUID, bind it to your listener instance, and then call  StartAdvertising.

Guid btUuid = Guid.Parse("6f5b75e8-be27-4684-a776-3238826d1a91");
RfcommServiceProvider provider = await RfcommServiceProvider.CreateAsync(RfcommServiceId.FromUuid(btUuid));

StreamSocketListener listener = new StreamSocketListener();
listener.ConnectionReceived += Listener_ConnectionReceived;

await listener.BindServiceNameAsync(provider.ServiceId.AsString(), SocketProtectionLevel.BluetoothEncryptionAllowNullAuthentication);

provider.StartAdvertising(listener);

Do note that before you call the StartAdvertising, you could also use the SdpRawAttributes method from RfcommServiceProvider to gain access to the SDP attribute dictionary, and use it to add your service specific SDP attributes.

Any incoming connection is then received with the ConnectionReceived event handler. In it you simply take the StreamSocket from the argument, store it, and read/write data to it.

void Listener_ConnectionReceived(StreamSocketListener sender, StreamSocketListenerConnectionReceivedEventArgs args)
{
 StreamSocket socket = args.Socket;
 ...
}

 

Receiving Bluetooth connections in Background

Full example for background usage  is available under BLETestStuffWindows at my GitHub repos. And it is called BluetoothAppBG.

For background processing you need to implement a background task. The general idea is explained in  Create and register a background task article, then to handle Bluetooth connections, you need to use specific trigger. And in this case it is RfcommConnectionTrigger.

With the RfcommConnectionTrigger, in general you would only need to set your own GUID to it, but as with the RfcommServiceProvider, you can also add you own custom SDP attributes, the only difference here is that you don’t get access to a Dictionary instance, instead you would need to write the attributes into an instance of IBuffer. For reference implementation, do check the getsdpRecordBlob function that starts at line 131 in BackgroundManager.cs file.

Creating the trigger part of the code then looks like this:

Guid btUuid = Guid.Parse("6f5b75e8-be27-4684-a776-3238826d1a91");

RfcommConnectionTrigger trigger = new RfcommConnectionTrigger();
trigger.InboundConnection.LocalServiceId = RfcommServiceId.FromUuid(btUuid);
trigger.InboundConnection.SdpRecord = getsdpRecordBlob(serviceName, serviceDescriptor);
backgroundTaskBuilder.SetTrigger(trigger);

Then once there would be incoming connection, then in your IBackgroundTask, you would get the Run  called with taskInstance.TriggerDetails set to instance of RfcommConnectionTriggerDetails. And with that instance, you could simply use the Socket method to gain access to StreamSocket instance for the connection, and use it for reading & writing data.

 
RfcommConnectionTriggerDetails details = (RfcommConnectionTriggerDetails)taskInstance.TriggerDetails;
 if (details != null)
 {
 socket = details.Socket;
 writer = new DataWriter(socket.OutputStream);
 reader = new DataReader(socket.InputStream);
 } 

Connecting to the service from Android App

If you would want to connect to the Bluetooth service running inside Windows 10 device from android, you could use any Bluetooth example (that works with Bluetooth Serial communications), for example the bluetoothchat available via SDK examples.

All you need to do, is to change the GUID to be the one used by Windows 10 service, and then, as the Windows side coding expects to get lenght of the message before the actual message starts, you would need to modify the sending part to do that in android. And with the SDK example, I did this by adding four lines of code into the write() function defined in BluetoothChatService.java file:

mmOutStream.write(((buffer.length >> 24) & 0xFF));
mmOutStream.write(((buffer.length >> 16) & 0xFF));
mmOutStream.write(((buffer.length >> 8) & 0xFF));
mmOutStream.write((buffer.length & 0xFF));

 

 

 

 

BLE device Simulator

I just started working for a project where I need to implement simple application for custom BLE device. Basically what I would need to do, is to read characteristics values, show them in the UI and store the data in a file.

This sounds pretty easy, however, the BLE device is currently under development, and I would not be getting one for the next couple of weeks.

I’m really eager on getting started with it, and as the task for the app is really simply to find a device, connect to it and to monitor & store values from it, I got thinking that maybe I could make a simple simulator that would fake the device.

And as I do have Android device which is supporting full peripheral mode, I indeed can get it done rather easily. Basically what I need to do, is to check what kind of stuff my Polar H7 is advertising, and then simply doing the same with my own BLE advertiser.

The result project is called HeartBeatSimulator and as always, its open source project available under my GitHub repository.

Note that, if you use the examples used with my Using BLE GATT services with Windows 10, you would need to remember that Windows does use caching when pairing Bluetooth devices, thus first do remove any existing paring for the device and then do following:

  1. Do run the simulator in android device first (and let it run in background while doing pairing)
  2. Open Bluetooth settings in the android device
  3. Switch to Windows device, and open Bluetooth settings. Do enable Bluetooth if its disabled
  4. Start pairing from Windows device, and if there is multiple entries shown for the device, so pair all of them (The right BLE device would be the one that asks you to supply 6 digits ID on the android side, so remember to fill it right)
  5. After successfull paring, start the Windows application (the one that reads the values from the simulator)
  6. Search devices, and Select the simulator device to connect to it.
  7. With first time usage, the Windows device will ask permission to use the BLE simulator device, and you need to accept this.
  8. After this, the data from the simulator should be updated to the Windows app normally.

With the Foreground Example I made earlier, I did notice that for some reason the call to ReadClientCharacteristicConfigurationDescriptorAsync  inside the HeartbeatEngine does harng, thus before I get it fixed, you should simply comment out the line. Anyway, do make sure you call the WriteClientCharacteristicConfigurationDescriptorAsync, since if its not handled, the value changed event’s will not be delivered to your app.

Using BLE GATT services with Windows 10

Windows 10 brings nice update on the apps front by introducing Universal Windows app . With it you can create one app, and run it in all Windows 10 platforms, including phones & laptops.

To get started with development for it, you could first fetch the Windows-universal-samples  that allows you to get your old device update to Windows 10. I personally used my really old 920 for development & testing, simply because I do have access to few of special Developer device versions of them.

The Universal Windows app examples are for some reason not including any examples for using BLE GATT services, thus I decided to make one. Of course there is older 8.1 example, which indeed could be used as a base.

Additionally Windows 10 brings a new API that be used with BLE GATT services, and that is GattCharacteristicNotificationTriggerDetails, which can be used for monitoring single Characteristic while running in background.

Reading BLE characteristics values in Foreground

With Windows 10 devices, in order to connect to any BLE device, you need to pair them first. Thus you need to go to the Bluetooth settings, enable the Bluetooth. And then find the device you with to communicate with and get it paired. Which after you can find the device in your own code by searching for it with DeviceInformation.FindAllAsync():

 var devices = await DeviceInformation.FindAllAsync(GattDeviceService.GetDeviceSelectorFromUuid(GattServiceUuids.HeartRate));
 
if (devices != null)
{
   foreach (DeviceInformation device in devices)
   {
       // add device into your UI here
   }
}

After we have paired with the device, it will be shown in the list even when it’s out of range, thus it’s advisable to actually to monitor whether we have connection, before attempting to read any values, though you can of course set value changed event handlers for characteristics, but those won’t be called if you don’t have connection anyways.

Do note that it appears that the device is handling the connection, thus from your application point of view, you simply find then device service, and then you check the status of the connection to that device, and then simply subscribe for connection events, which will inform you if we get any changes on connection. From coding point of view it looks like this:

service = await GattDeviceService.FromIdAsync(device.Id);
DeviceConnectionUpdated((service.Device.ConnectionStatus == BluetoothConnectionStatus.Connected), null);
service.Device.ConnectionStatusChanged += OnConnectionStatusChanged;

Then to get the characteristics from the service, you simply use the GetCharacteristics() function:

characteristic = _service.GetCharacteristics(GattCharacteristicUuids.HeartRateMeasurement)[0];
characteristic.ValueChanged += Oncharacteristic_ValueChanged;

 

And to make sure you’ll get notifications for the characteristics value change, you do need to make sure Characteristic notifications are enabled by making sure that characteristics configuration is set to Notify:

var currentDescriptorValue = await _characteristic.ReadClientCharacteristicConfigurationDescriptorAsync();

if ((currentDescriptorValue.Status != GattCommunicationStatus.Success) ||(currentDescriptorValue.ClientCharacteristicConfigurationDescriptor != GattClientCharacteristicConfigurationDescriptorValue.Notify))
{
await _characteristic.WriteClientCharacteristicConfigurationDescriptorAsync(GattClientCharacteristicConfigurationDescriptorValue.Notify);
}

After this, you are basically done. Then you just need to implement handling the events and updating your UI with the values. Full example for the foreground reading for heartbeat characteristics can be found under my github repo at: BLETestStuffWindows/HeartbeatFg

Background Characteristics monitoring

I also have full example for the background monitoring available under my github repo at: BLETestStuffWindows/HeartbeatBg, The Uis for both examples are exactly the same, but the usage of the BLE Api’s are quite different.

The main changes made for the foreground example were:

  1. The MainPage in addition to finding the device, also monitors connection to it. This is to make sure that we only attempt to display values for devices we actually have connection to.
  2. The HeartbeatPage gets the heartbeat value via progress event handler from background task, thus its simply listening the progress events, and showing the value in the UI controls
  3. We needed to remove any characteristics value monitoring etc. from the BLE engine, since the value reading is now handled in background. Only connection monitoring parts are still used in MainPage.

Additionally we needed to add new functionality

  1. There is now 3 modules in the app. In addition to the main application, there is background service part, and Common module which is hosting parts that needs to be accessed from both foreground & background parts.
  2. MainPage is also doing Background task registration. UnRegistratering is handled in App class (when going back from HeartbeatPage)
  3. When we launch the app while having active background task, the start page is changed to be the HeartbeatPage, since we don’t need to select any device for showing the values anymore. This has effected the navigation design in app class.

The Background registration  is handled in BackgroundManager class. This class also implements the functions for setting & removing progress events handlers needed by HeartbeatPage.

In MainPage we use the lighter version of the BLE engine to find the characteristic we want to monitor in background, then we use BackgroundTaskBuilder class to register it with the instance of GattCharacteristicNotificationTrigger:

BackgroundTaskBuilder backgroundTaskBuilder = new BackgroundTaskBuilder();

backgroundTaskBuilder.Name = HeartbeatMonitorBackgroundTaskName;

backgroundTaskBuilder.TaskEntryPoint = HeartbeatMonitorBackgroundTaskEntryPoint;

backgroundTaskBuilder.SetTrigger(new GattCharacteristicNotificationTrigger(characteristic));

BackgroundTaskRegistration backgroundTaskRegistration = backgroundTaskBuilder.Register();

The framework then should handle keeping the connection to the device, until we remove the background task. And any change in the characteristics value should cause our background task to be executed.

In the background task, we then simply get the value of the characteristic from the taskinstance, and set it to the Progress variable, and if we have the progress event handler set, our UI will be informed on the newly gotten value.

GattCharacteristicNotificationTriggerDetails details = (GattCharacteristicNotificationTriggerDetails)taskInstance.TriggerDetails;
byte[] ReceivedData = new byte[details.Value.Length];
DataReader.FromBuffer(details.Value).ReadBytes(ReceivedData);

HeartbeatMeasurement tmpMeasurement = HeartbeatMeasurement.GetHeartbeatMeasurementFromData(ReceivedData);

taskInstance.Progress = tmpMeasurement.HeartbeatValue;

In the example implementation for the background task, I also have additional ways on informing the user of the values:

  1. ToastNotificationManager is used to make notifications to inform user of the cancellation of the task (so they could click the notification and re-start the task from the UI app)
  2. TileUpdateManager is used to update the main tile of the application with the heartbeat value, so the user can see what the last value we got was without opening the UI app.
  3. System.Launcher.LaunchUriAsync() is used to start the UI application directly when we have the heartbeat value outside the defined range.

 


					

AltBeacons with Android

I was reading my good friends Juhana’s Blog about BLE BEACONS ON WINDOWS 10 and was wondering on how would I do these beacons with Android.

Juhana did his BLE work alongside my another friend Tomi, thus I of-course did check Tomi’s Blog, and got his example for Advertising & scanning for Altbeacons from his WORKING WITH CONSPICUOUS DEVICES article. This example implementation was then used later on to check that it can see my android advertisements in right way, as well as that my android implementation would see the windows device Altbeacons  as well as could decode the scan record right way.

Juhana had nice illustrator showing the advertisement package used with AltBeacons, and it looks like this:

AltBeaconScanRecord

Juhana also mentioned in his article that “DataType value should be a constant 0xFF as specified in the AltBeacon protocol specification”, and after seeing this I went to check what the Generic Access Profile says for the type 0xFF.

As the 0xFF type means ‘Manufacturer Specific Data’, and it’s the first type on the package, we simply need to get an advertisement scan record, which only has the manufacturer segment and we are done.

With the BluetoothLeAdvertiser API, to take out any unnecessary data from the scan record, we need to:

  • With AdvertiseSettings call setConnectable with false. This makes the beacon not being connectable, also it makes sure the flags part in scan record is not added.
  • With AdvertiseData call both setIncludeDeviceName and setIncludeTxPowerLevel with false value to make sure they are not included in the scan record either

The to set the data for the scan record, we can use the AdvertiseData::addManufacturerData(int manufacturerId, byte[] manufacturerSpecificData) function. Note that the manufacturer id is encoded to the scan record automatically by the API, and it takes 2 bytes. Rest of the data is supplied by byte buffer and its written directly to the scan record, thus you can use your own encoding inside it.

Following picture illustrated how the data is written into the scan record. The Length & Type fields are set by the API and are both taking 1 byte. The manufacturerId is set from the value you supplied with addManufacturerData call and it takes 2 bytes. The green part was also supplied alongside the call for addManufacturerData function, and it can be up to 27 bytes long (31-1-1-2).

AltBeaconScanRecordAndroid

I did also quick search for any library that could be used with altbeacons and did find some. In general they looked a bit too big for the simple task needed here, and I wanted to have code that is really minimalistic.  Also wanted to have codebase that could easily be modified to support different data schemes used for the 27 byte payload.

In general, only codes that would be needed here, are involved on encoding & decoding the scan record data accordingly.

The resulting example is called AltBeacon  and you can find it from github under my BLETestStuff repo 

I implemented both encoding & decoding in the AltBeaconFactory.

Generally the code is rather strait forward, for example the encoding part simply puts the variables stored in AltBeacon  class instance into a byte array according to the type and encoding used in the scan record.

public static  byte[]  getAdvertDataFromBeacon(AltBeacon beacon) {
    byte[] ret = new byte[24];
    //beaconCode
    int beaconCode = Integer.decode("0x"+ beacon.getBeaconCode());
    ret[0] = (byte)((beaconCode >> 8) & 0xff);
    ret[1] = (byte)(beaconCode & 0xff);
    // id1 2 - 17
    String tmpString = beacon.getId1().replaceAll("[\\s\\-]", "");
    int index = 2;
    for(int i = 2; i <= tmpString.length(); i = i + 2){
        if(i <= 32){
            ret[index] = (byte) (Integer.decode("0x" + tmpString.substring((i - 2), i)) & 0xff);
            index++;
        }
    }
    //id2
    int id2 = beacon.getId2();
    ret[18] = (byte)((id2 >> 8) & 0xff);
    ret[19] = (byte)(id2 & 0xff);
    //id3
    int id3 = beacon.getId3();
    ret[20] = (byte)((id3 >> 8) & 0xff);
    ret[21] = (byte)(id3 & 0xff);
    //refRSSI
    ret[22] = (byte)beacon.getRefRSSI();
    ret[23] = (byte)beacon.getManufacturerReserved();
    return ret;
}

The BeaconScanner is implemented in a way that it is using BluetoothLeScanner with devices having Lollipop or never, and with older devices its using the BluetoothAdapter::startLeScan() API.

One notable issue I run into with Marshmallow was the new security scheme google implemented with it. With BLE beacons, when the code is run with device shaving the Android 6.0 (i.e. Marshmallow) you are required to add ACCESS_COARSE_LOCATION into then manifest file.

On top of having the permission defined in the manifest, you also need to ask users permission for the capability. For implementation details check the MainActivity  class implementation.

The code was tested with API level 18 (Jellybean), 19 (KitKat), 21(Lollipop) as well as 23(Marshmallow) devices. Here’s video showing how the app runs on these different devices:

The distance shown on the screen is of course a bit off because I did not have any good way on getting really good value for the measured power sent with the scan record.

But the video does show clearly the fluctuations on the distance calculated, and this is caused by the measured Rssi, thus to get the distance fluctuation out, we should use some kind of averaging the Rssi over time, instead of the last measured value alone.

Thirdly, it can be seen that the values are not really accurate, thus the measured distance should not be shown as meters, instead we should use zoning.

Good example of the zoning can be seen with splitting the distance into three zones with iBeacons:

  • Immediate: Within a few centimeters
  • Near: Within a couple of meters
  • Far: Greater than 10 meters away

 

Quick research on BLE discovery

We have been having the Thali postcard demo ready for quite a long time, and while we been using it, it has been really easy to see that the Wi-Fi Direct discovery is really not up to par. It does work, time to time, but most often it just fails to discover some peers, or occasionally it simply takes way too long it to discover anything. On top of this, it very often also stops seeing peers that are still around, and marks them unavailable, thus making it impossible to communicate with them.

One alternative approach for the discovery would be to use BLE, and as the data we require to exchange can’t be included in the scan record data (it has total limit of 31 bytes), we can’t use simply beacon approach, but instead we would need to use a real connectable peripheral, which would serve the data via the GATT service.

As I was going over this earlier in summer, we did not really have a good view of what hardware requirements BLE would pose. In essence BLE central (i.e. scanning) has been working since API level 18, though some might argue that it’s crappy with API level 18, and you should actually use API level 19.

But with peer discovery, we do also need to be able to advertise our presence, for this we need to get the BLE Peripheral to be supported. This is then supported from API level 21 (Lollipop), and same time there was new scanning API released, bringing better battery usage as well as supposedly tons of improvements. So far with my simple tests, I have not seen any big differences between the BLE scanner APIs, though with KitKat API you do need to create own parser for the scan record.

So from the API point of view the requirement is that the device must support Lollipop. Then we soon discovered that not all Lollipop devices support being BLE Peripheral. For example LG Nexus 5, did have support for the API in early pre-launch versions of the Lollipop for it, but with 5.0.1 at least it was removed. Supposedly the reason being that the HW with it does not support being BLE peripheral & central same time.

For Thali discovery the requirement indeed would be that the device must be able to simultaneously to advertise its presence as well as to seek other devices advertising their presence. Then quick look into the Bluetooth specs made it very clear, it indeed appears that Bluetooth 4.1 lets any device be both a peripheral and a hub at the same time. Small change in terminology there, but indeed that’s what was needed. Thus the revised device specs was that all devices must support Lollipop as well as have Bluetooth 4.1 HW.  Back in the summer when I made my first list, I did only find less than 10 devices which fitted into these specs, luckily the numbers of devices is increasing rapidly.

Then as I now knew which devices I would need to have, the next task was to create a nice set of codes for advertising & discovery, and to do some testing to see how well the API behaves with the intended usage. The results app can be found at Github.

The apps logic got a bit complicated. I was having good amount of issues with BLE connection & characteristics reading, basically when you design your logic, you should remember:

  • Only one request of any kind can be active at any given point
  • If there is an error, you must have a short break before re-attempting anything, otherwise the device might get into a state it cannot recover from the error anymore.
  • With some devices, it appears that you should run all stuff in one thread (UI thread might be preferred)
  • Don’t ever call gatt.close() right after gatt.disconnect(). This will cause internal error inside the API, instead simply call disconnect, and do close inside the onConnectionStateChange callback

And even when remembering all of these you’d end up getting occasional 0x85 / 133 GATT_ERROR errors, which after you should have a break (30-60 seconds should be fine) before you do anything else with BLE.

The logic for the discovery with the example app, is to

  • first find the peer via BLE scanning, and then
  • to do connection to it, which after we
  • request characteristics, find ours and
  • read the characteristics value

Important issue here is that we should always start connection freshly discovered peer. Since if we store a discovered peer for later connections, and we get to try connections after it has moved out of reach, we would be getting connect error, which would require us to have a break on API usage, making out discovery to work slower.

Then with initial tests I had 3 devices meeting the specs doing full discovery rounds once in a minute. And did determine that all worked just fine. There were errors with some connections, but re-tries did succeed eventually and I could run the tests for hours without having problems.

Then to make things more interesting, I did also boot up 5 additional phones which were doing only the discovery (they did not have Bluetooth 4.1, and thus could not run the advertising API), and strait away I started to see more error with the original 3 devices. And after running the tests for several days, I did not find any ways on making the logic to work reliably.

In all tests, after just couple of hours of running, one or two of the 3 devices doing both advertising & discovery, started to have error situations where they never recovered. Most often the connection request to the GATT returned the 0x85 / 133 GATT_ERROR, and I did not find any nice ways I could get the app to recover and start succeeding on the discovery.

The issue appears not to happen if we do less work on BLE, thus I changed the logic this in mind. First thing to do here is to save all discovered devices data, and if we see the same device again, we simply use the old data, which indeed reduces the work we do with BLE.

With android the BLE address is changed every now and then, the actual value depends on the manufacturer, but its guaranteed to change after x seconds. Also the BLE address will be changed every time we stop & start the BLE advertising, thus, if we need devices to re-do the full discovery, we have a nice mechanism for getting it done.

Then I decided that same time as I do read for the characteristic on the other device, I could also do write and give it my discovery data. That way it would not need to connect me to get my data, and thus it would be doing less work on BLE. I’m calling this Discovery Push via BLE.

Then I realized that I actually have some bytes I’m not using with the BLE scan record. I only have the flags there, and the full length UUID. Thus I actually had enough space left for adding 6 byte long Bluetooth address in it. Which would allow me to make insecure Bluetooth connections for exchanging the discovery data.

Thus I revised the advertiser code to include the Bluetooth address as a service data, and in discovery parts I added code that does insecure Bluetooth connection to the advertising device, if the BLE discovery would fail. Effectively also reducing the work we do with BLE.

With insecure Bluetooth connections, I could also do discovery push, similarly as I do with BLE characteristics. The problem here is than that we are using the BLE address as an identifier for the peer. With BLE we can see the address inside the callback that gets called when we issue the write. But with insecure Bluetooth, we of course only see the ‘normal’ Bluetooth address in the receiving end. And as I have not figured a way on reading my own BLE address so far, we couldn’t use this method for pushing discovery, unless we can deal with the lack of BLE address.

The following image illustrates the resulting logic for the discovery.

AndroidBLEDiscoveryFullStuff

With the final logic, at least I could run the discovery codes for hours without any of the devices going to any irreversible bad state, though I only had 3 devices acting the both advertiser & scanning parts, and 5 additional devices doing the scanning. Thus more testing with higher number of devices would be needed before I would trust the discovery code. Anyway, currently I’m not working on thali, thus that work would be needed to get done by somebody else.

 

First results from BLE battery consumption tests

UPDATE: After going through the results, I did notice that the Nexus 6 device I got, was indeed broken. It basically has something badly wrong with it, and even when there it nothing running it will drain its battery rather quickly, thus the results shown in this page are indeed faulty.

I was running the  BleGgOne test app for last four days to get some idea on how long battery life we could expect if we would be running BLE advertising and scanning on the devices all times.

Note that I only had one device capable of doing the BLE service advertising, thus effectively with the tests one device was advertising all times, and the other device was scanning continuously. The tests were conducted in space where we did not have other BLE devices active while the tests were running.

Nexus 6 has 3220 mAh battery and the Nexus 5 used as the client connecting to the service advertised by Nexus 6 has a 2300  mAh battery.

There was two rounds of testing. The first round was called Full Test, and with it the Nexus 6 was advertising one service. Nexus 5 then connected to the Nexus 6 and with 60 second timer, it did first read for a character that had 100 bytes of data, and then also write of 100 bytes to that same character.

Logs from tests showed that the read/write events were reported to happen once a minute from both ends during the whole testing. Also the connection was active for the whole duration of the tests, thus there were no disconnection events happening during testing.

In the second round, the client side code for connecting to the server was commented out. Meaning that there were no connection made between the devices, nor any read or write events happening. The Nexus 6 in this test was simply advertising the service, and the Nexus 5 was simply scanning and finding the service.

From logs it was also determined that with the 20 minute log period, the scanning resulted 4300 results in both tests, giving the scanner average of 3.6 Scan results for each second.

The results are summarized in following graph:

 

BLE_Battery_first

Following table shows the times when different threshold values on battery status were reached:

 Battery % / time reached N6_AdvertOnly N6_FullTest N5_ScanOnly N5_FullTest
90 3,45 3,45 5,51 5,85
80 6,86 6,52 9,63 9,97
70 9,97 9,29 13,72 14,07
60 12,71 12,36 17,83 18,17
50 15,44 14,74 20,91 21,96
40 18,17 17,50 25,37 26,41
30 20,91 20,24
20 23,34 22,99
10 26,06 25,71
0 28,45 27,76

 

Quick calculations from the values gives drop of 10% of battery charge happening with Nexus 6 after 2.8 hours, and with Nexus 5 its around 4 hours.

Surprisingly there appears to be no real difference between the device being just scanning / advertising, when compared to the battery consumption with devices that are connected and exchanging data.

As this test was only conducted with one device, we could say that the values are indicative at best, and to get clear picture of the battery consumption, we do need to make loads more testing with different devices.

 

Quick view to JxCore Thali plugin architecture (android part)

I’m going to have my summer vacation after this week, and wanted to make it a bit easier to anybody who wants to check my codes to actually understand what each parts are doing there.

And to do this I decided to draw some graphics depicting how different classes are interacting and being handled in the system.

The first part the the main view for the plugin from the part where JxCore android plugin meets the android library for Thali:

Main_stuff

The JxCore plugin is using the library through a helper class called BtConnectorHelper, which is the sole entry point  to the library. From library side, the sole entry point is the BtConnector class.

The BtConnector class then is handling two issues. It should know when to start/stop advertising & scanning. The codes for the task is handled in BtConnector_Discovery class. BtConnector_Discovery class then has its internal logic to find peers and uses simple three function interface to pass the results to the BtConnector.

The other task BtConnector has to do, is to handle connectivity to the peers. This is handled in BtConnector_BtConnection class.  BtConnector simply starts/stops the class instance, and the instance will simply tell the BtConnector when outgoing connection failed, or when it did manage to get fully connected socket connection to the remote peer.

Then lets have a look into these two classes and their internal works. The BtConnector_Discovery is rather simple.

BTConnector_Discovery

The WifiServiceAdvertiser, is simple started/stopped when the instance of BtConnector_BtConnection is created/deleted. Then the WifiServiceSearcher will have its own logic to find peers and services implemented in them, and use the three function interface to notify the instance of what it has found.

The BtConnector_BtConnection class then looks like this:

BTConnector_BtConnection

 

In essence, there is always one BtListenerThread active, and ready for any incoming connections. The one possible BtConnectToThread instance is created when there is tryConnect() call originating from BtConnectorHelper class. Error situations on incoming connections are never reported outside BtConnector_BtConnection , they simply re-start the listening process.

Once either of the previously mentioned class instances has managed to create connection successfully, they pass the connected socket to the BTHandShaker class instance.

This  instance will use its own BTHandShakerSocketThread for reading/writing to the socket to handle quick handshake. The handshake makes sure that the connection is fully working, as well as it is used for supplying the discovery data in cases where the device we are connecting to, does not have discovery data for the incoming peer yet.

The Connected callback function of the BtConnector will only be called if the BTHandShaker gets its handshake successfully done.  And once this happens the BtConnector will give the BtConnectorHelper fully connected socket, and also identify whether it was incoming connection we accepted in out listening thread, or whether it was connection we started.

The following picture then illustrates what happens in the BtConnectorHelper side for the connected socket:

BtConnectorHelper

If the socket was for incoming connection, then there is a new instance of BtToServerSocket  created for it. There can be multiple instance of BtToServerSocket , and currently the implementation does not pose any limits for the number of them, but do note that the Bluetooth performance likely drops when there are more connection to be handled.

For outgoing connection there can only be one instance of handler available at any moment. The handler is then instance of BtToRequestSocket  class.

Both BtToServerSocket  and BtToRequestSocket  are then having two instances of StreamCopyingThread class. One for copying incoming stream and one for copying data to outgoing stream.

And as the name suggest, the BtToServerSocket  is connecting the incoming Bluetooth socket streams to the NodeJs TCP/IP server streams, and the BtToRequestSocket is catching the NodeJs TCP/IP client request and connecting it to the Bluetooth streams.

Basically this is just really brief overview of the implementation, there are of course loads of things that happens when the codes are run, anyway, I do hope this article would help on understanding the internal structure of the android implementation.

Starting my BLE investigation

I finally got myself a device that supports BLE advertising. First issue on getting one was to figure out what devices are actually supporting the feature. Even though there are claims that some Android devices can be used as BLE peripheral while having only Bluetooth 4.0 chip on them, I would strongly suggest getting a device that supports Bluetooth 4.1. Currently only confirmed 4.0 device I know would be Samsung S5, though I have not used it myself. Nexus 5 also was claimed to work with earlier preview versions of Lollipop, but I also found discussions suggesting that while it was used to advertise, it could not be used for scanning same time, supposedly that was the main reason it’s not supporting the feature with official Lollipop versions.

So from API perspective the requirement is Lollipop support, and from hardware it would be Bluetooth 4.1 support, and with quick search my list of supporting device got to look like this:

  • Nexus 6,
  • Nexus 9,
  • LG G4
  • LG G Flex 2
  • HTC One M9
  • Samsung Galaxy S6
  • Sony Xperia™ M4 Aqua

 

The first device I got was Nexus 6, for historical reasons I have several Nexus 4 & 5 devices, so it makes nice continuity to have one 6 as well. I should be getting Sony XPeria M4 Aqua also soon, and as the device is loads cheaper than Nexus, as well as even its having smaller battery, it does have double standby time ( according to ubergizmo:). It also has less RAM, and slower CPU, thus it likely make nice device to do tests and compare the results with the Nexus.

And to get started I had to get some codes. I was surprised that I couldn’t really find any nice full code samples that would do scanning (and parsing the scan info), connecting & character/descriptor read and write. Thus I made my own full example to create & tests helper classes I can re-use in later tasks. The resulting example is stored at DrJukka/BLETestStuff/MyBLETest . The implementation is still work on progress, and current limitations include for example, that only string-type values are supported, and that only non-reliable write is implemented.

I alsready used the classes developed with the example in other project DrJukka/BLETestStuff/BleGgOne. This project was used to test that we can run the app in background and still do advertisement & scanning. I also made second version of it, to test that we can have more than one background app doing the advertising.

The result was that Yes, we can do advertising in the background, and Yes we can do scanning in the background as well. Then as I only have one device to test, I did not get to see whether I could do both same time. I’ll need to wait my XPeria device to finalize the testing.

And for the question that “can we have multiple services advertising in background” the answer is also Yes.

As with the BLE, each time you start advertising, the device is advertised with new virtual-mac-address. So if you stop and start a service, it will get new address which the client device must use (using the old one will cause 0x85 error to appear)

When using two services same time, these is something similar happening, basically a new device advertisement with new virtual-mac-address will be visible to the clients. But also the first services virtual-mac-address is still visible, and in this case both of them are valid and can be used for connecting to the device.

But when you connect either one of them, you will be getting the all services from the device. So it does not matter which one you would be using for connecting, you would still get service advertised by both of them.

Continuing the Dns-Sd Txt record size measurements

Earlier research with Wi-Fi Direct and Dns-Sd Txt record size measurements  suggested that there maximum size that is less than 1k buffer for the Dns-Sd records. With the Delivering discovery data via Wi-Fi Direct UPnP I also noted that at least with UPnP the buffer is actually shared with all apps, meaning that its actually not a limit of one service advertisement, but actually its limiting the combined size of all service advertisements active on the device.

I did quick tests and determined that, indeed this its same with the Dns-Sd records, and there also the latest added service can easily make the previously set advertisement to never actually be advertised, effectively making the service invisible for anybody looking for it.

Additionally it was earlier determined that the instance name does effect on the data amount. With my investigation I also determined that also the length of the service type string will affect the limits posed for the instance name.

As I never really did finalize the instance name limit investigation, I decided to get it done now. I was using the good old WifiTestPower  app, and modified the service type to be just 10 characters long. I was using Nexus 5 devices. Two of them had KitKat (4.4.4 and 4.4.3) and two had Lollipop (5.0.1 and 5.1.0).

The result showed that the max limit is not really set by the receiver (as I earlier thought), but actually its set by the limits posed on the sending device.

With KitKat I determined that when using 10 character long Service type string, then the usable limit appears to be 100 characters.

Actually I can see 101 very often, but its causing loads of time when the service discovery after finding one service will timeout without finding anything, and as this does not appear to happen when using 100 characters long instance name, I would put the limit there.

Funnily enough, when I give the WifiP2pDnsSdServiceInfo empty txt-record array, then even when I’m not getting the DnsSdServiceResponseListener called for instance names over 101, I do get the DnsSdTxtRecordListener called with 103 characters long instance name, and indeed the fullDomainName argument is having the full instance name included.

That all said, with the current knowledge of the topic, if the service type is 10 characters long, I would still put the maximum length of the instance name to be 100 characters.

With Lollipop, if the service type is 10 characters long, the limit appears to be 255 characters limit works with both interfaces, and if exceeded no discovery data is seen to arrive either of them.

So if you want to get the discovery work with and between both KitKat and Lollipop then with 10 character long service type we could use up to 100 characters long instance names.

Note then that if we also take account the less than 1K total limit, I would assume we could safely assume that in one device we can have 6-8 services advertised before the system starts dropping them from the actual advertisement.