Controlling 433 MHz Power Outlets with openHAB2

I’m currently building a home automation system based on the incredibly powerful openHAB 2 platform. We already have a few remotely switchable 433 MHz power outlets by the manufacturer Brennenstuhl in our home which we currently switch using the provided remote controls. I was wondering whether we could control them from the OpenHAB platform as well, and indeed found a way to achieve this.

My openHAB 2 instance does not run on a Rasberry Pi, but on a dedicated Ubuntu Server. If your platform is Raspberry, your hardware setup and configuration might be different, but still I think this article will be useful for the OpenHAB binding configuration.

Hardware

First, I looked for a suitable device capable of sending and receiving 433 MHz signals. I ended up with a nanoCUL device connected via USB. There are many DIY nanoCUL kits available on the internet that you can assemble yourself, but there are also pre-built nanoCULs available. I chose the latter and ordered an assembeled nanoCUL USB device including an antenna and a USB adapter. It looks like this:

nanoCUL with antenna and USB adapter

OpenHAB Binding

After some research I found a suitable binding to integrate nanoCUL with openHAB: it is called Intertechno Binding. It is an older v1 binding, and is not displayed in my Bindings list after the installation (even when activating the Include Legacy 1.x Bindings option). But it works nonetheless.

To configure the nanoCUL, edit the file services/culintertechno.cfg and add the following configuration:

1
2
3
device=serial:/dev/ttyUSB1 
baudrate=38400
parity=NONE

You have to adjust the device (in my case /dev/ttyUSB1) to the device matching the nanoCUL on your system. To find out which device it is, I used a script I found in this stackexchange answer.

Binding the Device using a Unique Identifier

After a few reboots I discovered an issue: sometimes, the nanoCUL was bound to /dev/ttyUSB1, other times to /dev/ttyUSB0. This lead to errors and conflicts in openHAB. To solve this problem, I used a device path like the following in services/culintertechno.cfg:

1
device=serial:/dev/serial/by-id/<id of your nanoCUL>
You can find the device ID of your nanoCUL using
1
ls -la /dev/serial/by-id/
But when I started openHAB2, the following error occurred:
1
org.openhab.io.transport.cul.CULDeviceException: gnu.io.NoSuchPortException
I found out that this can be solved by adding the device path to the java startup options of openhab2. In my case, these can be configured in /etc/default/openhab2:
1
EXTRA_JAVA_OPTS="-Dgnu.io.rxtx.SerialPorts=/dev/serial/by-id/<id of your nanoCUL>"
After adding the option and restarting openhab2 with sudo service openhab2 restart, the error disappeared and now the system has the correct device association after every reboot.

Item Configuration

The tricky part was to find out which codes to send to switch the power outlets on or off. After long research, I found this FHEM Wiki page which finally helped me to figure out the codes. The power outlets are configured with DIP switches like the following:

The first 5 switches identify the logical group of power outlets. The remote control that comes with the power outlets has the DIP switches 1-5 only (excluding A-E). A-E identifies one of 5 power outlets in the group.

To derive the code to be sent from openHAB, you just have to translate the switch states into a sequence of 0 and F, where 0 corresponds to “switch up” and F corresponds to “switch down” (I did not get why “switch up” is encoded with a lower value than “switch up”, but anyway this is how it works for me). So for the switch states shown above, the code is

0F00FF0FFF

This is the basic code to address a specific power outlet in a specific group, where the first five digits encode the group and the last five hex digits the outlet in the group. To control whether the outlet should be turned on or off, one the following two codes has to be appended: FF = ON or F0 = OFF. So in conclusion, to switch on the above outlet, the complete code is

0F00FF0FFFFF

and to switch it off, the code is

0F00FF0FFFF0

In the item configuration, this is added as follows:

1
Switch MyOutlet_B "My Outlet B" {culintertechno="type=raw;commandOn=0F00FF0FFFFF;commandOff=0F00FF0FFFF0"}

And that’s it folks, it works like a charm for me 🙂 I hope this post will be useful to others who want to integrate their 433 MHz power outlets in openHAB 2.

Generating EMF Models from XML Schema Definitions (XSDs)

In this blog post I will show how to generate models for the Eclipse Modeling Framework (EMF) out of an XML schema definition. EMF is a powerful framework which allows you to create Java classes corresponding to the XML schema types, code to load XML documents to Java models and code to serialize Java models back to XML again. As an example, I will use MusicXML schema definitions.

Installing Required Features

To work with EMF and XSD schemas, you need to install the following features in your Eclipse development environment:

  • EMF – Eclipse Modeling Framework SDK
  • XSD – XML Schema Definition SDK

You can check for already installed in the About dialog of your Eclipse installation. In case the features are not installed yet, go to Help -> Install New Software… and choose the update site corresponding to your Eclipse version. For example, for my 2019-09 Eclipse the update site is http://download.eclipse.org/releases/2019-09. Search for the two features and install them.

Creating an EMF Project and Importing the Model

If you don’t have a project already, create one using New -> Other… -> Eclipse Modeling Framework -> EMF Project. Specify a project name and click Next. Several model importers should be proposed.

Select XML schema in the list (it should have been installed with the XML Schema Definition SDK) and Click Next. The following page appears:

Click Browse File System… and select the XSD you would like to import. I recommend not to select the option Create XML Schema to Ecore Map. The Generator model file should have an appropriate name already, otherwise you can change it here. I changed the capitalization slightly. Click Next.

On the next page you can specify the file name of the generated ECore model file. It should align with the generator model file name you just specified. Click Finish.

You should end up with a new project containing the folder model. It in turn contains the imported data model in an ecore model file (MusicXML.ecore). It also contains a generator model file named MusicXML.genmodel. If you want to make any adjustments to the data model (classes and attributes/references), this can be done in the ECore model. However, since this an imported model this should not necessary in our case. Below is a screenshot of the imported model:

Adjusting the ECore Model

The only adjustments we need to do for now in the ECore model are:

  1. Right-click the Musicxml package below the root element and choose Show Properties View
  2. Change the Name and the NS Prefix to musicxml (note the lower case m). This is important because this will become part of the java package we will generate.
  3. Set the NS URI to http://www.musicxml.org/xsd/MusicXML

Adjusting the Generator Model

The generator model gives us control about how and where the Java classes for our model will be generated. Select the package below the root element and open the Properties view. Adjust the following settings:

  • Base package: enter the common Java package name prefix which should be put in front of all classes/interfaces/enums to be generated, e.g. org.myapp. Note that the ECore package name will be appended to this prefix automatically. For example, if you use the base package org.myapp and your ECore Package name is musicxml, the code will be generated in the Java package org.myapp.musicxml.
  • Prefix: this is the class name prefix used for EMF-specific classes such as factories and utility classes. I propose to change this to a CamelCase identifier you would put at the beginning of a Java class name. Example: the prefix MusicXML will generate class names such as MusicXMLFactory, MusicXMLPackage, MusicXMLResourceImpl.

Generating the Java Code

Now it’s time to generate the java classes. In order to do that, right-click on the MusicXML package and choose Generate Model Code.

After the operation finishes, you will see lots of interfaces/classes/enums generated in the src folder of your project:

If you want the source code to be generated in another source folder such as src/main/java, this can be adjusted when editing the properties of the root object of the generator model.

Loading an XML File

Now that we have our Java code, we can use the EMF infrastructure to load an XML file to a java model. Loading a file in EMF typically involves:

  1. Creating a ResourceSet
  2. Registering appropriate resource factories in the resource set
  3. Loading a resource by specifying an URI

The following code assumes that a file is loaded from disk, but you could also specify internet URIs instead of the file URI:

1
2
3
4
5
6
7
8
9
10
11
12
13
public static Resource loadMusicXMLFile(File musicXMLFile)
{
    ResourceSetImpl resourceSet = new ResourceSetImpl();
    resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put("xml", new MusicXMLResourceFactoryImpl());
    resourceSet.getResourceFactoryRegistry().getExtensionToFactoryMap().put("musicxml", new MusicXMLResourceFactoryImpl());
 
    // disable DTD resolution since it fails for MusicXML files
    Map<String, Boolean> parserFeatures = new HashMap<>();
    parserFeatures.put("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
    resourceSet.getLoadOptions().put(XMLResource.OPTION_PARSER_FEATURES, parserFeatures);
 
    return resourceSet.getResource(URI.createFileURI(musicXMLFile.getAbsolutePath()), true);
}

For most scenarios, the first three lines and the last line would be sufficient to load an XML file to a Java model. For MusicXML, I had to tweak the XML parser configuration a bit, because it tried to load MusicXML DTDs from a server which failed. Since we don’t need DTD validation anyway, I disabled the parser feature to load external DTDs. The map with the parser feature settings in turn has to be put as value for the load option key XMLResource.OPTION_PARSER_FEATURES, and EMF will take care of forwarding the parameters to the XML parser. Call getContents() of the returned resource to access the java model representation of the loaded XML file.

If you want to save a MusicXML java model to an XML file, basically use the same code as above, but call resource.save() instead of resourceSet.load().

That’s it 🙂 I hope this blog post illustrated how easy and powerful XML to Java object mapping can be when using EMF. Of course this can be done with any correctly structured XSD, not just with MusicXML.

When Logic Destroys Your Audio Files

This post is about a serious bug in Logic, which causes audio files to be damaged. The symptom of this bug is the following error message when opening a previously perfectly working project: One or more audio files changed in length.

Background

This is how I encountered the bug: we had recording sessions with our band and had completed the first session. In the studio, we could listen to all tracks without any problems. After some time, when our sound engineer opened the project again, he encountered the error message above for some files. After listening through the projects, he discovered that a large number of audio clips could not be played back anymore. We had one backup of the project we had made right after the recording session, but unfortunately it contained the same corrupted files. This meant that we had to repeat the recording session 🙁

We had a few more recording sessions, after which we made 3 or 4 backups right away. After we had recorded a whole album, the issue occurred again. The files were corrupted on all backups again, although we could listen to the tracks in the studio without problems. Consequently, the data was corrupted after the recording and before re-opening the Logic project.

Then, our sound engineer noticed the following: when he opened a project for the first time, logic reported 4 corrupted files. When opening the same project again, suddenly 24 files were corrupted. This leads to the conclusion that Logic itself is responsible for destroying the audio files.

Conditions under Which the Bug Occurs

According to internet sources, the bug occurs under the condition that the computer utilizes differently formatted hard disks, i.e. disks which are not all formatted with Mac OS X Journaled. The files are damaged when logic is closed, which means that even if the project works perfectly before closing the application, there is no guarantee that it will work when opened again.

We experienced these problems with Logic 9, but internet forum posts suggest that it can also happen with Logic Pro X. If someone can confirm or has any updates, please feel free to comment.

Error Analysis

Logic destroys the audio files in seemingly random order, e.g. in a sequence of 79 audio files recorded for one track the files with numbers 43, 48, 56, 57, 58, 59, 65, 74 and 79 were corrupted.

For a more thorough analysis, I compared working files with corrupted files using a Hex editor, in which each single byte in the file can be visualized in hexadecimal representation. The first bytes of an intact wave file look like this:

For a detailed description of each byte, refer to this page. In short, these are the contents of the wave file header:

  • Bytes 1-4: RIFF chunk descriptor
  • Bytes 5-8: chuck size (total number of bytes in the file after this block)
  • Bytes 9-12: format (in this case WAVE)
  • Bytes 13-16: fmt-subchunk header (contains fmt )
  • Bytes 17-20: subchunk 1 size (in this case 16 for PCM)
  • Bytes 21-22: audio format (1 = PCM)
  • Bytes 23-24: number of channels (1 = Mono, 2 = Stereo, etc.)
  • Bytes 25-28: sample rate (e.g. 44,100 Hz)
  • Bytes 29-32: byte rate: number of bytes required to store 1 second of audio for all channels (= sample rate * number of channels * bits per sample / 8)
  • Bytes 33-34: block align: number of bytes required to store one sample in all channels (= number of channels * bits per sample / 8)
  • Bytes 34-36: resolution in bits per sample, e.g. 8, 16 or 24 bits
  • Bytes 37-40: data chunk header (contains data)
  • Bytes 41-44: number of bytes representing the raw audio data
  • Bytes 45ff.: raw audio data

Now let’s have a look at a destroyed audio file:

If the bug occurs, Logic fails to write the wave header correctly. Instead, the file contains only zeroes in the first 44 bytes, which is exactly the length of the wave header. The good news: the raw audio data, starting at byte 45, is still intact (note that the hex editor starts counting bytes at index 0).

If such a corrupted wave file is opened, logic can’t read the header and assumes a default 8 bit setting, which leads to a misinterpretation of the audio data. Additionally, the length of the file will also be misinterpreted, because a wrong sample rate is assumed.

Repairing the Audio Files

As a preliminary fix, you can restore the destroyed files by copying a wave file header (i.e. the first 44 bytes) from a correct file (with matching sample rate and resolution) to a corrupted file in a hex editor.

Update August 29, 2019: It was confirmed that this also works for AIFF files. In this case, the first 512 bytes have to be copied. Thank you very much to Sawyer Wildgen for sharing this!

A wave file header specifying a sample rate of 44.100 Hz and 24 bit resolution starts with bytes similar to these (in hexadecimal representation):

1
52 49 46 46 5B 89 3E 00 57 41 56 45 66 6D 74 20 10 00 00 00 01 00 01 00 44 AC 00 00 CC 04 02 00 03 00 18 00 64 61 74 61 6B 5C 3E 00

However, one potential issue now could still be that the (sub)chunk sizes (bytes 5-8 and 41-44) are not correct, but most audio editors don’t check these values. If you want to correct these as well, make sure that you use the correct little endian representation for these byte groups. This means the byte order is reversed. A complete example is given below.

The formulas to calculate the correct values are:

  • chunk size = <file size in bytes> - 8
  • data chunk size = <file size in bytes> - 44

Integer to Little Endian Hex Conversion

Example: Converting the number 44,100 to a little endian hex number:

  1. Convert number to hex using a scientific calculator or an online converter such as this one. The result is: AC 44. Note that this result comprises two bytes and is encoded big endian (most significant byte first).
  2. Make sure the result is padded to the correct byte size. If the field in the header is 4 bytes, we have to add two zero bytes at the beginning: 00 00 AC 44
  3. Reverse the byte order: 44 AC 00 00. The result is now little endian (least significant byte first), as required by the wave header specification.

To confirm, you can open a working wave file with 44,100 Hz sample rate in a hex editor and check bytes 25-28, which will contain 44 AC 00 00.

Using Wave Recovery Tool to Restore the Wave File Headers

Because quite many files were damaged in our case, I did not want to fix all wave headers manually. Therefore, I developed a program which can fix the wave files all at once. Wave Recovery Tool is available on github and is published under the terms of the GNU General Public Licence.

Conclusion

This post reveals a serious bug in Logic, which can potentially destroy hours and weeks of hard work. Fortunately, the data can be restored completely either manually or using a Wave Recovery Tool I developed. I seriously hope that this bug will be fixed soon or is already fixed in recent versions of Logic.

Notes Regarding AIFF Files

In this blog post, I demonstrated the issue by means of wave file header structures. The same can be done for AIFF files, however the header structure is more complicated. The good news: I extended Wave Recovery Tool and now it is also possible to restore AIFF files under certain conditions.

Replacing org.eclipse.equinox.ds with org.apache.felix.scr in Eclipse Product Builds

In recent Eclipse versions, the declarative service implementation Equinox DS was replaced with Apache Felix service runtime components. As a result, my product build with an Eclipse 2018-12 target platform using Tycho failed with the following error:

Unable to satisfy dependency from toolinggtk.linux.x86org.eclipse.equinox.ds 4.2.0.201810261013 to osgi.bundle; org.eclipse.equinox.ds 1.5.200.v20180827-1235

To solve this problem, I had to update my .product file. Open it with the product editor and switch to the Configuration tab. In the middle, bundle start levels can be configured. Most likely your product file still contains an entry for org.eclipse.equinox.ds here. This has to be removed and replaced with an entry for org.apache.felix.scr with start level 2. This can easily be achieved by clicking the Add Recommended button on the right hand side. My resulting configuration looks like this:

Product Start Level Configuration

If everything is set up correctly, you can build your product again. Happy coding!

Adding Custom Splash Screens for Eclipse Products

Today I tried adding and configuring a custom splash screen to a custom Eclipse application, and found a number of pitfalls, which is why I decided to write a blog post summarizing the required actions.

Adding the Splash Bitmap

The splash image must be saved as a bitmap file named splash.bmp with 24 bits per pixel and should have dimensions of approximately 500×330 pixels. Save this file in the root of your product-defining plugin (which is usually named tld.domain.product.rcp). Specify this plugin in the Splash section of your .product file.

Enable Progress Reporting

To enable progress reporting on your splash screen, an .ini file for customizations must be defined. This is normally done in your RCP plugin, which contains a plugin.xml file declaring the extension org.eclipse.core.runtime.products. Under this extension, your product is defined. The product node typically contains child nodes with properties of your product, such as the name of your application (property appName). Here another property has to be added with the name preferenceCustomization and the value plugin_customization.ini. Create this file in the root of your RCP plugin if it does not exist already. To enable progress reporting for your splash screen, add the following line:

org.eclipse.ui/SHOW_PROGRESS_ON_STARTUP=true

Adjust Positions of Progress Bar and Progress Message

Now comes a pitfall: In older Eclipse versions, the positions of the progress bar and progress message could be adjusted in the product file. The following UI is provided for that:

However, in newer Eclipse versions the values of this UI are not used anymore! In order to configure the positions, properties have to be added in the previously mentioned plugin.xml file of your RCP plugin as children of your product node.

The positions are added in the form of comma-separated lists in the following format:

x, y, width, height

As an example, consider the following entries:

1
2
<property name="startupProgressRect" value="0,290,498,10"/>
<property name="startupMessageRect" value="5,310,493,15"/>

The coordinates are relative to the top left corner of the splash screen image. Additionally, the color of the progress message text can be changed with the property startupForegroundColor. The value is a hexadecimal color code with two hex digits for red, blue and green, respectively. For example, black font color can be specified with the following code:

1
<property name="startupForegroundColor" value="000000"/>

Relevant documentation can be found here:

Eclipse Enablement Expressions for Files with Specific Extension

While developing a UI plugin for Eclipse, I wanted to activate a specific context menu entry only for files with a specific extension. Because it took some time to figure the solution out, I decided to share it for others who want to implement this.

This example demonstrates the enablement for a launch shortcut, but of course this can be adapted to other extensions with enablement support, too.

One or More Files with Specific Extension

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<extension point="org.eclipse.debug.ui.launchShortcuts">
  <shortcut class="tld.domain.plugin.ui.MyLaunchShortcut"
            icon="icons/icon.png"
            id="tld.domain.plugin.ui.launchShortcut"
            label="My Launch Shortcut"
            modes="run">
    <contextualLaunch>
      <enablement>
        <with variable="selection">
          <count value="+"/>
          <iterate operator="and">
            <adapt type="org.eclipse.core.resources.IFile">
              <test property="org.eclipse.core.resources.extension"
                    value="ext">
              </test>
            </adapt>
          </iterate>
        </with>
      </enablement>
    </contextualLaunch>
  </shortcut>
</extension>

The enablement is based on the selection, which should contain at least one element (count=+). Then the resources are iterated and adapted to IFile if applicable. Then the file extension is checked (in my example the extension ext, i.e. all files with the file name pattern *.ext are permitted). Note that the test properties must be given with a namespace (e.g. extension is wrong, whereas org.eclipse.core.resources.extension is correct).

One or More Files with Multiple Valid File Extensions

Another use case is to allow multiple file extensions. In this case, an additional <or> element is required as child of the <adapt> instruction as shown below:

1
2
3
4
5
6
7
8
9
10
11
12
13
<enablement>
  <with variable="selection">
    <count value="+"/>
    <iterate operator="and">
      <adapt type="org.eclipse.core.resources.IFile">
        <or>
          <test property="org.eclipse.core.resources.extension" value="xml"/>
          <test property="org.eclipse.core.resources.extension" value="xsd"/>
        </or>
      </adapt>
    </iterate>
  </with>
</enablement>

Added June 18, 2019:

Multiple Files with Multiple Valid Extensions

The following enablement tree will show the element if more than one file .xml or .xsd file is selected simultaneously. Note the corresponding count expression (1-, which means 1 to unbounded, where 1 is excluded.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<enablement>
  <with variable="selection">
    <count value="(1-">
    </count>
    <iterate operator="and">
      <adapt type="org.eclipse.core.resources.IResource">
        <or>
          <test
                property="org.eclipse.core.resources.extension"
                value="xml">
          </test>
          <test 
                property="org.eclipse.core.resources.extension"
                value="xsd">
          </test>
        </or>
      </adapt>
    </iterate>
  </with>
</enablement>

Single File with Specific Extension or Container (Project / Folder)

This will enable the UI element if a file with a specific extension, a project or a folder is selected:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<enablement>
  <with variable="selection">
    <count value="1">
    </count>
    <iterate ifEmpty="false" operator="or">
      <or>
        <adapt type="org.eclipse.core.resources.IFile">
          <test
                property="org.eclipse.core.resources.extension"
                value="mcl">
          </test>
        </adapt>
        <instanceof
                    value="org.eclipse.core.resources.IContainer">
        </instanceof>
      </or>
    </iterate>
  </with>
</enablement>

Also see the Eclipse Documentation for a full list of available properties and the Platform Expression Framework documentation.

Renaming git-managed Eclipse Projects

During longer development projects it sometimes becomes necessary to change the overall bundle structure, involving renaming packages and project/bundle names. For git-managed projects this requires one additional step in order to keep bundle names and their physical git location in sync. This blog post summarizes the steps for the most common refactoring actions.

Renaming Package Trees

Eclipse provides excellent support for renaming packages recursively. Right-click the root package of a bundle and then choose Refactor -> Rename. This brings up a dialog in which you can enter the new package name. This dialog also provides the option Rename Subpackages which will rename the whole package tree recursively. Note that all references to the classes in the renamed packages will also be updated.

Renaming Projects/Bundles

Renaming a project basically requires the same steps as renaming packages. Right-click a project, choose Refactor -> Rename and enter the new project name.

Now comes the interesting part: When renaming a git-based project the change will not be reflected in the filesystem of the git repository. Instead, only the <name> tag in the .project file is adjusted. You can verify this in the project properties in the Resource section (compare Path and Location).

To change to pyhsical git location, you have to specify the new location using the Refactor -> Move dialog. Here you can adjust the actual folder name in the git repository.

moveGitLocationAfter that, you can commit all changes in git. The history of the moved project will be preserved. I hope this will help you re-structuring your projects.

Getting Rid of Rattling Sounds in a Dual 1210 Record Player

Vintage record players like the ones produced by the German company Dual are still popular today because of their excellent workmanship and sound. I have the privilege of owning one and using it regularly. Unfortunately it began making an annoying rattling sound lately. This is how I repaired it: The first challenge is opening the record player. I found this video explaining how this is done for Dual record players. The following picture shows the inside of the device. The mechanical constructions for the automatic movement of the arm are absolutely amazing!

DualPlattenspielerMitPfeilThe rattling sound was caused by the motor, which is located in the lower center of the picture. In order to get rid of the noise, you have to disassemble the parts surrounding the motor and oil them well. Afterwards the record player should work smoothly again. If this does not help, it might be necessary to disassemble the motor itself as explained here.

Creating Full Backups of your Android Smartphone Without Root Access

In this blog post I will explain how you can create a full backup of your Android smartphone, even if you don’t have root access.

“Become” a Developer

The first step is to activate the developer options in your system settings menu. To do that, locate the build number in your device info menu. This is usually located somewhere under Settings -> Device Info. Once you located it, you have to tap the build number seven times in order to unlock the developer features of your phone.

Enable USB Debugging

Once you unlocked the developer mode, there is a new menu in your settings called Developer Options. Find this menu and enable USB Debugging.

Install the Android Debug Bridge

The next step is to install a tool called adb (Android Debug Bridge). It is part of the Android SDK (Software Developer Kit). The problem is that the current version as of the time of writing (1.0.32) is unusable due to a bug (see bug report). As soon as this bug is fixed, you can dowload the SDK Command Line Tools from the official download page or install a current version using Homebrew.

Update March 9th, 2017: Tested again with version 1.0.36 (Revision 0e9850346394-android), and the backup works. However, an app blocked the backup process. After uninstalling the app, the full backup was successful.

Connect Your Device

After you installed a working version of adb, you can check the connection between your device and your computer. To do this, you need to locate the adb executable file in your SDK installation or your extracted folder. Open a terminal and type the following command:

/path/to/adb devices

where /path/to/adb must be replaced with the absolute path to the adb executable. Mac users can drag and drop the adb file from the Finder into the terminal in order to insert the complete path automatically. This should result in an output similar to:

List of devices attached 
3204226a921e227d    device

Make the Backup

If your device is connected, you can perform the backup with the following command:

/path/to/adb backup -all -apk -shared

This starts a backup process including all apps (-all), APK archives (-apk) as well as the internal and external storage data (internal memory and SD cards, -shared). If you don’t need all of this data, you can skip the correspondent parameter.

After issuing the command, you will be asked to confirm the backup operation on your smartphone. You can enter a password to encrypt your backup, which I recommend. Note that the backup process can take a long time depending on how much data is persisted on your phone. The process should result in a large file called backup.ab which is stored in the current folder (if you did not navigate to another directory: in your user home folder). If you want the backup to be stored elsewhere, you can supply the parameters -f <path> specifiying the target file path.

If the file is empty or contains only 41 bytes (when using no password) or 549 bytes (when using password) then you have a buggy version installed and need to get an older version (see previous sections).

Restoring Backups

Backups can be restored using the command

adb restore /path/to/backup.ab

Installing Python 3 and music21 using Homebrew and pip3

In this blog post I’m going to explain how the Python library music21 can be installed in conjuction with Python 3 and its dependencies matplotlib, numpy and scipy on Mac OS X. It can also be used as a tutorial for installing any other Python libraries/modules as well.

The Problem

Initially, on my system there were two parallel Python 2 and Pyton 3 installations. The music21 installer chose Python 2 as default installation target. In order to use music21 in conjuction with Python 3, I tried to install it using the command

1
pip3 install music21

which worked fine. However, when I tried to use the plotting capabilities of music21 an error occured due to the missing modules matplotlib, numpy and scipy. When trying to install matplotlib issuing

1
pip3 install matplotlib

the following error occurred:

SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.

Installing Python 3 using Homebrew

My final solution to this problem was to set up a new Python 3 installation using  Homebrew. This is done by installing Homebrew (if you haven’t got it yet):

1
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Python 3 is installed using the command

1
brew install python

Note: Previously, the formula name was python3 but was renamed to python as Python 3 is now the official default version. The old version can be installed using brew install python@2.

If you already have Python 3 installed, Homebrew will not be able to create symlinks to the python binaries since they already exist. To overwrite the existing symlinks (and thus to set the Homebrew Python as default interpreter for your system) you have to execute this command:

1
brew link --overwrite python

Now, symlinks to the new python installation are created under /usr/local/bin.

By the way: python updates can now be installed by simply executing

1
brew upgrade python

Adjusting the $PATH Variable

In some cases it might be required to tweak the settings of the $PATH environment variable, namely if the old Python 3 installation is still preferred by the system because of a $PATH entry with higher priority. To check if this step is necessary, type:

1
which python3

If the output is /usr/local/bin/python3, you can proceed to the next section. Otherwise, check the contents of your $PATH variable with this command:

1
echo $PATH

which might look like this:

/Library/Frameworks/Python.framework/Versions/3.4/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/subversion/bin:/sw/bin:/sw/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:

As you may notice, the old interpreter entry /Library/Frameworks/Python.framework/Versions/3.4/bin precedes /usr/local/bin. To give priority to the new Python 3 interpreter, change the order of the paths, ensuring that /usr/local/bin precedes other python paths:

1
export PATH=/usr/local/bin:[more path elements here]

This command changes the $PATH settings for the current shell session only. If you want to make the path adjustments persistent, add the command to the file .bash_profile in your user home folder. It is also possible to reuse the current value of the variable:

1
export PATH="/usr/local/bin:$PATH"

After that, our Python 3 installed with Homebrew should now be the default system interpreter. Verify this with

1
which pip3

which should echo /usr/local/bin/pip3, which is in turn a symlink to the Homebrew cellar (the place where Homebrew installs modules/packages).

Installing the Dependencies

Now you should be able to install music21 and the dependencies using pip3:

1
2
3
pip3 install music21
pip3 install matplotlib # this will install numpy automatically
pip3 install scipy

I hope this will help you to install a clean music21 environment. No go have fun with musical analysis and plotting 🙂