Enabling UART on Raspberry Pi

Today I enabled UART on raspberry Pi. The process followed for this can be found at:

http://www.instructables.com/id/Read-and-write-from-serial-port-with-Raspberry-Pi/

This was required in order for the ST NFC driver to work on RPI3. Obviously the idea was to use the UART driver in order to test stuff.

Making pi upload pics on bootup

Following the task from this week, I had to combine almost everything I did into the ultimate task. The goal was to make pi start taking pictures just as it boots up and then upload them consecutively to a web server. Of course in order to upload pictures the pi must be connected to the internet. And the script to take pictures and upload them should also be working. Since all of these things were working individually, it was time to put all of this together.

One would think that making pi take pictures on boot up is easy as all that is needed to done is call the script from rc.local and things would fall into place. So I thought. But that was not the case. As it turns out that writing commands in the rc.local only ensures that the commands are run. It does not ensure that the commands are run successfully. This of course was problematic as I had no way to ensure the network was up before pi could start clicking pictures. The whole thing was a synchronization mess. I need to find a better and more clean way of doing this.

Applying driver patch for NFCST

Today I build the environment using Buildroot for ST NFC after applying the patch prepared to Buildroot. There were two ways to do this. The first one was the straight forward one, where I would take a Linux source, apply the patch and then have buildroot compile it. But researching a bit I found out that there is way in which we can apply custom patches to the Linux kernel downloaded by buildroot. All we need to do is enable that option and give the path to the kernel patch. And so I did.

Structure Packing

Run the following piece of C++ code and observe the output:

#include <iostream>

struct something {

        char c;

        int i;

        short s;

};

int main()

{

        std::cout << sizeof(something) << endl;

}

The output will be 12.

Now run the following program:

#include <iostream>

struct something {

        int i;

        short s;

        char c;

};

int main()

{

        std::cout << sizeof(something) << endl;

}

The output will be 8.

Now that you have run both versions of the same program on a computer, run them in your head. Both programs are doing the same thing. They are printing the size of a struct. The struct contains the same type of variable in both cases. And yet the size of the struct comes out to be different. The structs are almost identical with the only difference being the order in which the variables are declared inside the struct. So the moral of the story is that the order in which variables are declared in a struct determine what the size of struct is going to be. But you should ask your self that why in God's good name should the declaration order matter to size, after all the same number and type of variables are used and hence the size of memory they occupy should also be the same. Well clearly this is not the case. To understand why, you need to learn this first. 

Data alignment! It was fun wasn't it? A direct consequence of this sort of alignment of data in C (and C++) is the way in which compound data types like structs are given memory. Remember how C likes its variables to be stored at memory addresses that are a multiple of their size? Well C doesn't discriminate, the same rule applies to structs. So if a struct is of size S, then any instance of this struct will be stored at an address that is a multiple of S. This is very intuitive as you already know why this is required. The issue is the fact that since a struct is a compound data type, its size can be any arbitrary number of bytes depending on what variables it holds as members. The whole alignment thing works only because the data types were allowed to have certain convenient sizes i.e. 2, 4, 8 ... This means that if structs are allowed to have any arbitrary size then aligning them according to their size will make arbitrary sense (i.e. it will work well sometimes and sometimes it will seem stupid).

Hence it is imperative that the compiler do something to make sure that the structs also end up getting convenient sizes i.e. multiples of 4. The rule that C has to ensure this is rather simple, The size of the struct is always made to be a multiple of the size of the biggest member of the struct. In the example above, the struct something has int i as its biggest member with size 4, so the struct size will also be a multiple of that. How does the compiler achieve this? By using structure packing.

To understand this, you need to be introduced with some nomenclature. X from now on is the size of the biggest member of the struct. Boundary refers to any multiple of X following the address where the struct memory begins. So some struct object begins its memory at 20 and X is 8 for that struct, then the boundaries are: 28, 36, 44, and so on.

Structure packing is a technique (or a policy) in which if adding a new member of the structure to memory crosses any boundary in the struct, then before adding that member, the leading (proceeding) bits are left as padding and the new member is made to start immediately after the said boundary ends. Finally if after allocating memory to all members in this way, the struct is shy of the next boundary by a few bytes, then those bytes are also used as padding just to make the structure a direct multiple of X and end always end up on the boundary exactly.

Consider the first example of the struct something. The value of X is 4 (size of int i, the biggest member of the struct). Adding the first member of the struct, char c crosses no boundaries and the size of struct reaches 1 (which is the size of char). Then comes the turn of int i. int i is 4 bytes in size, adding it to the memory of struct will make the sum 5. This is problematic, int i is crossing the boundary of the first multiple of 4 i.e. 4. In this case the compiler will decide to allocate 4 bytes to int i starting from byte number 5 only and the bytes in between (2, 3, 4) will be used as padding. So int i will start from 5 and will go till 8. The variable left to be added now is short s which is 2 bytes in size. Adding it to 8 does not cross any boundaries so no proceeding padding is needed. But since the final outcome comes at the total of 10, an extra two bytes are padded just to make the size be 12 i.e. a multiple of 4. This way the final size comes out to be 12 bytes, out which 7 bytes are the variables and 5 bytes are just padding. That is some extravagant wastage don't you think?

Now moving onto the second example. The value of X is still 4. The first variable to be added is int i, adding it gets the total to 4. This covers the first boundary entirely. Then comes short s. Adding this takes the total to 6. So far no addition has cost the total to cross any boundaries. Lastly it is required to add char c. c is 1 byte in size. Adding it to the struct makes the total memory equal to 7, which as you may notice still does not cross the next multiple of 4 i.e. 8. Hence so far we are golden, no padding is needed. The only thing left to do is to add one byte of padding in the end to take the total to an even 8. And vola, that is the final size of this struct, with 7 bytes of actual data and a frugal byte for padding.

You see now, how rearranging members can make a difference in size of the struct. One might say that okay saving a few bytes is cool but it isn't very significant. That may be true for moderate size programs on moderate size computers, but if either the program is very big or the computer on which it is to be run is low on memory, then these little alterations can add up to amazing benefits.

The next time you are writing a struct, may be keep in mind to arrange the members in such a way that the small ones come together so that they can be fit inside a single chunk and padding is needed to the least.

Another important thing to know here is that the standard does not guarantee that the padded bytes are to be zeroed, they can be anything, from zero to any arbitrary value. It is the programmer's duty to play around with these carefully. Assuming that the members in a struct are all packed together continuously is plain foolish.

Testing picture taking scripts on Pi

Raspberry Pi has a pretty cute way of integrating a camera into it. The command to do this is called raspi-still. My task for today was to test a script that took pictures continuously and uploaded them to a web server. The script was ready but it had some bugs as it was not working as expected. I could not tell what the bug is as I was encountering such things for the first time. But after searching on the Internet about how each command written in the script worked, I came to form a decent understanding of how the script worked. This put me in a position change the script around and find the bug. And so I did.

I had to add some commands in order to make the script fool proof. The script worked like a charm after that.

Applying STNFC driver patch on Linux source

In order to test whether the Patch created for the STNFC driver works, I cloned a fresh source for the Linux kernel from github and then applied the patch to it using the git apply command. And then verified if the changes made by the patch were accurate and apt. After solving many issues I was  able to create a patch that could singularly make all the changes required by the STNFC driver to work to the Linux kernel

Data allignment in C

I am fan of travelling. But just not as a hobby or an activity I'd like to take, instead as an analogy to understand and hence communicate some efficiency concerns in computers. I am going to do this again.

When one has to travel and that to frequently then what you'd expect the number of items of their luggage to be? As less as possible right? A traveller (at least a wise one) would want to pack things in (say) one bag so that carrying stuff isn't as prominent a pain as it otherwise would be. Now assume that on a single trip, one is allowed to take only one bag. Considering the trip costs money and more importantly, time, one would want to take this trip once only. Packing your stuff in two separate bags would be stupid as then you'd have to travel twice in order to complete the transfer.

The idea is to keep things packed in such a manner that one would not require more bags to carry them. You want only one bag or at the most, a number of bags that you can carry at once.

A program travels too, more frequently then any human in any imaginable circumstance. And while travelling, a program also has to carry stuff from memory to the CPU, or back. As such you'd want your program to carry the whole data belonging to the same variable once, in one data cycle. A 64 bit CPU can carry 64 bits at once. So if you have 65 bits of data in memory then that will force your CPU to take two trips in order to fetch this data entirely. But this is okay as no altercation would make this transaction more efficient, CPU can only carry 64 bits in parallel.
Another interesting peculiar feature of CPUs is their habit of considering only those chunks of memory that start from an address which is a multiple of the word size. So if the word size was X then the CPU would only fetch memory chunks starting from 0, X, 2X, 3X, ... This means that if some thing is placed in such a way in memory that it is not wholly a part of any of these chunks, then the CPU will have to make two trips in order to fetch it and then will have to perform some bit shifting in order to restore the proper value of the thing being fetched as well.

Data types in C (with exception of char) have even bit sizes. Their size is always a multiple of 2. More importantly, except for chars, all other primitive data types in C are either a multiple or a factor of the word size on most architectures. Most modern architectures have a word size which is a multiple of 4, so this means that almost all primitive data types in C should also be a multiple or a factor of 4 bytes in size. You won't see a data type in C that is primitive and has a size of 6 bytes or 10 bytes or 14 bytes with the exception of long double. The size will be either 2, 4 or any multiple of 4.
When a compiler has to store a variable of a particular type, the starting address available to store the variable can be anything (depending what and how things were packed before). If the compiler just went ahead and stored the variable exactly at the address that was available then that could lead to some inefficiencies in getting that variable back from memory. For example consider that the variable size is 4 and the starting address available to store the variable is 18, then if the storage was done simply, the variable will reside in memory from the address 18 to 22. This will lead to the same problem described in the earlier paragraphs. Since the variable will be residing in two separate memory chunks (that start from a multiple of 4 and are 4 bits in size), the CPU will have to take two trips in order to fetch it completely and then will have to perform some bit shifting on the partial values of both the chunks in order to retrieve the proper value of the variable. Now ask yourself, wouldn't it be nicer if this variable was stored at address 20 to 24 even if that meant leaving two empty bits at 18 and 19? Of course it would, as now the CPU could simple fetch the variable in one go and would not require any manipulation of bits in order to get the proper value of the variable.
For this reason the C language enforces some rules that determine how the variables of various sizes are to be stored. The general rule is that if a variable is of size S then it must be stored at an address that is a multiple of S as well. Since the the size of variables (as mentioned above) align (mostly) with the word size of machines, this enforcement ensures that no variable has to be split between two memory words.

Git send email for STNFC

I had studied and used git send email before but had left it there only. Today I was required to use it in order to create a patch for STNFC driver. This made many changes to the Linux kernel so the size of the patch I was required to create was clearly bigger than any patch I have ever created.

I needed to brush my skills over git send-email in order to be able to do this. It helped that when I studied it I create a document explaining everything required. I studied up that document and went ahead with the work in hand.

First of all I made all the changes by hand. Commited the changes and then created a patch from it. But of course the patch created was not in the right format as checked by the scripts given in the Linux kernel source. So I had to add signed off to the commit and also a description of the commit in order to make things perfect. Finally I had a single patch that could hopefully incorporate all the changes required to run the driver.

Configuring kernel for ST NFC support

Making a driver work in the kernel is not just about having to add the source and compile the kernel with it. The driver may depend on a lot of things in order to work properly. And all of these things need to be enabled. Same was the case with the STNFC driver. First of all it used to transport buses (not together of course), SPI and UART. So in order to use them, proper support was required in the kernel source.

Today I spend the whole day figuring out how to enable these and what exactly is required for the driver to work. Finally I was able to enable all the necessary things.

Spartan Programming

You all have seen the movie 300 which makes you enough of an expert to understand this post. It would help if you knew some programming as well but it's just an after thought so don't beat yourself for it.

Remember how Spartans are almost anti-fancy in their attire and yet a menacing force of nature? Well it so happens that there is a culture in programming inspired almost from this principle i.e. be as effective as possible without using too many tools. This culture is aptly named Spartan programming.

A programming work can be deemed as being Spartan based on the following criteria:

1. Horizontal complexity: A program shouldn't be too nested. More nesting there is, the more chances you have of clearing your logic and making it more streamlined.
2. Vertical complexity: The length of the program should be as short as possible. No point in writing more code when the problem can be solved with something shorter and hence sweeter.
3. Token count: A token is any entity in a program that have individual meaning be it your variables, literals, keywords or anything that is allowed by the language and is part of your program. It is obvious that reducing token count without loosing meaning can ensure simplicity to your algorithm and hence can be more effective and scale-able.
4. Character count: Don't be too verbose with your comments and variable names. Be brief and to the point.
5. Parameters: It is always nice to refactor a solution into subroutines. A good subroutine should have a clear purpose and should be spartan in itself. One of the essential elements of ensuring this would be reducing the passing of implicit data to subroutines as arguments. You don't want to pass things that are available otherwise and can be derived from other parameters.
6. Variables: Don't overuse a variable in too many places. It is also considered a good practice to declare variables immediately before they are to be used and not all together at the top of your program.

In conclusion, spartan programming encourages frugal use of resources as a programmer. Another important feature is to make your solution robust just like a spartan. A spartan isn't invincible but it certainly should be tough enough.

Programming is about dumbing things down.

The more code you write, the more you wish you had written less.

What I mean by this is that after completing almost any programming assignment, one looks at ways in which the code could be optimized. Since optimization is a much more intellectual task, programmers like me try to confuse it with the size of code. I belong to a category of coders that think less the code more optimized the algorithm is. And you know what, for once I think I am on the right side of things in computing.

While being able to write crazy amount of code is a fantasy we all have but no one would question the beauty of an elegant solution to a tricky problem that saved day at the office. Computer scientists make a living out of telling dumb things (computers) to do dumb things (instructions) in order to achieve impressive feats (solve a Rubik's cube or simulate the universe, take a pick). What this process does is that it eventually breaks in a computer scientist the egoistic complexity a human has innately. They (I hesitate to include myself) tend to think about things from a more elementary point of view. Redefining problems in order to make the solution seem much more obvious,  classifying behavior in order to be able to compute it and being succinct in what they have to offer for the sake of being fast and not more omniscience.

An awesome algorithm may not be one that solves a hundred problems at once but rather an algorithm that solves a single problem simply. It is important for a coder to realize that computers can never be as smart as we are. Getting smart requires evolution and evolution takes time. So in order to be effective as a programmer one needs to learn to solve problems from a much more dumber perspective. Once we move being intelligent out of the equation we can start focusing on being less complicated. And once we are less complicated, we are less inclined to be wrong. And what do we call something that is not wrong? Right.

Write programs that can make sense to computers, after all they are the ones that have to run it.

UML: User Mode Linux

Linux is a name for a kernel. A kernel is a big piece of tough software that makes the wilderness of hardware behave for the comfort of the user. The user may run applications of varying tastes on the hardware without ever having to explain the billion gates inside of a computer, which way to let the current follow and to hold on which side. It is all taken care of the kernel.

But what if I told you that the kernel too can be run as an application. After all it is in the end a piece of software although a little firm. This idea isn't that alien of course. It kind of resembles virtualization.

You see, Linux being a kernel is meant to work on and with hardware. But if one wishes to run Linux in user mode i.e. like an application then it can't be given the kind of access it needs to the hardware. So what is the best that can done? Visualize an entire machine of hardware in software (simulate) and allow the application (Linux) to run the courses on it. It is important to note that the host for such activities must also be running on Linux.

It is an immediate given that this is fun. Running multiple copies of a kernel on machine simulates having multiple machines working independently of each other on a single big piece of hardware. Apart from being funny, this can also be very useful. The idea of having a kernel but not with unlimited power and almost no authority over your not so free and hence obviously precious hardware makes perfect sense for innovative development and kernel tweaking. You can use this technique to alter the kernel as much as you like without putting your computer in harm's way.

Even if you have some sincere activities planned, you can seek great aid from such virtualization. For example multiple servers can be hosted from a single machine this way. It can also be a great way to teach students about kernels and Linux.

All things said, this technique is not yet comprehensive. There are many leaps of development needed before it becomes more popular and hence starts gaining some developer attention as well.

Let's ID UUIDs

A UUID stands for Universally Unique Identity. Pretty heavy don't you think? To claim that something is unique in the entire universe is a bold move. Anyway, a UUID is a number representing the type of data stored in BLE servers, beacons and anything that uses Bluetooth and stores some data.

The way Bluetooth technology looks at data is centred around UUIDs. A UUID is used to classify data into certain types. It is just like a data type like your ints and floats of the world. The difference is that each type is a huge number and there is a huge number of these types.

Like in any programming language, same binary digits would mean different things depending on what type of data they are i.e. what is their UUID.

Now that you know what a UUID is, I want you to question who decides these UUIDs? The obvious answer would be the people that came up with the whole idea of UUIDs i.e. the Bluetooth SIG. This answer is true but not complete. As it turns out, there are two types of UUIDs that can be found:

One type is of the UUIDs that are 16 bits long and the other is of the UUIDs that are 16 bytes long. So you can imagine that the first type will contain a much lesser number of UUIDs.

The 16 bit UUIDs are provided by the standard. They were set at the time of inception of this specification or in consecutive years but only the SIG. A list of these UUIDs can be found at the Bluetooth website. Since these are provided by SIG, they remain constant. These are sort of like the built in data types supplied by a programming language.

The 16 byte UUIDs are kind of made up by various organisations and companies around the globe (or dare I say the universe) as they go along exploring newer areas and inventing new kinds of data crunched by Bluetooth devices. Once someone has an idea for a new type of data (a new profile), then they have to register that with the SIG and also provide the specification for it. This type will then be given a 16 byte unique number which will not be used by any other type afterwards. The larger length of this type of UUID make sense because the number of such applicants is enormously big. This whole thing is like making a new class and then turning it into a library for the rest of the world to use. The UUID is just a unique ID for your class/library.

Advertising using a BLE beacon in Linux

Beacons are a great way to send brief but important data to nearby scanning objects. They will be a force to be reckoned with in the future i.e. a world full of Internet of Things. In this post I will present a way in which you can advertise like a beacon from a Linux machine with attached or inbuilt Bluetooth low energy hardware. 

Since one expects beacons to advertise constantly, it is more likely that you'll be applying this concept to less consuming computers like your Raspberry Pi's or BeagleBone Black's. 

Let us get on with some primary installation stuff. Most of these things should already be installed on most systems but run these commands just for the sake of it:

sudo apt-get install libusb-dev 

sudo apt-get install libdbus-1-dev 

sudo apt-get install libglib2.0-dev --fix-missing

sudo apt-get install libudev-dev 

sudo apt-get install libical-dev

sudo apt-get install libreadline-dev

sudo apt-get install xclip

After this, you need to start your Bluetooth adaptor (assuming it is BLE supportive, find all about this here). Now lets get to business.

A beacons advertises. You know that. But the next defining thing about beacons is their inability to connect to any scanning device. A beacon just transmits data, it does not engage in a connection with any other BLE device. So the first thing we need to do is get your adaptor advertising in non-connectable mode:

sudo hciconfig hci0 leadv 3

sudo hciconfig hci0 noscan

The number 3 in the above command represents the non-connectable state. To advertise in connectable state use 0 instead and loose the successive "noscan" command.

Your adaptor should be advertising now. Now it is time to put some data in the advertisement. The data to be advertised has to be classified into a type. The beacon must also tell what type of data it is advertising. After all the thing to be advertised will be binary digits only but the way these bits are to be interpreted by the scanner depends on the type of data the bits represent.

The type of data in BLE specs is governed by the UUID which stands for Universally Unique ID. UUIDs are of different types (learn more about them here). The type of UUID we are going to use is a 16 byte UUID.

Since the UUID is 16 bytes long, it can practically be any random number. So go ahead make up any 16 bytes of imaginary data, write them up in hex like so:

63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5

The command used for sending out data is

sudo hcitool -i hci0 cmd 0x08 0x0008

But it takes a few arguments, one of which is the UUID that we have just covered, now lets look at the rest of the arguments.

The first byte in the argument list is the length of the entire packet to be sent. Lets just assume it to be XX for now, once we have figured out the entire packet then we can come back and fill in the length accordingly.

The rest of the packet is made of structures called AD structures. An advertising packet basically contains, the length field (XX for now) followed by any arbitrary number of AD structures each having a special purpose. An AD structure has the following format:

1 byte for the length of the structure following the length byte (assume this to be n)

1 byte for the type of the AD structure. There are certain types of these structures, each type representing a purpose.

n -1 bytes of data.

Our packet is going to have 2 AD structures.

The first AD structure is to set certain flags. Such AD structures that are used configure flags have the type 01. The value of the structure is the flag mask to be used to configure the flags. The flag mask is only 1 byte long and I am not even going to pretend that I know anything about what these flags mean. You're just going to have to trust me with the value and the value is 1A. So in total the length should be 02 as that is the number of bytes in this AD structure after the first length byte.

The next and last AD structure will eventually contain the payload of this packet which constitutes the value and UUID. We will calculate the length after all the other bytes have been accounted for in this structure. The type of this AD structure is FF. It represents Manufacturer information. The value of this AD structure contains the company ID followed by the payload. The company ID is a two byte code assigned by Bluetooth SIG to top semiconductor industries in the world. The code for STMicroelectronics for example is 00 30 (to be written in the reverse order in the packet). After the company code, there is a byte that represents the number of bytes remaining from here in out in this AD structure. The value for this is 15 in hex which equals to 21 more bytes. 16 of these 21 bytes are the UUID. Following the UUID is your 4 byte data which can be anything of your choice (I am going to use 00 00 00 00). The last byte in the structure is the Tx power with which this signal is to be transmitted.

Let me put this in order and then we can count the number of bytes and substitute that for XX.

XX 02 01 1A 1A FF 30 00 02 15 63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5 00 00 00 00 C8

So these are 30 bytes following one byte for length. Hence the first byte should be the hex representation for 30 i.e. 1E. Now, finally our command looks like this:

sudo hcitool -i hci0 cmd 0x08 0x0008 1E 02 01 1A 1A FF 30 00 02 15 63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5 00 00 00 00 C8

You can now scan the beacon from other BLE devices. I would recommend using the beacon scanner app for Android.

To stop advertising use the following command:

sudo hciconfig hci0 noleadv

Making pi upload pics after bootup

After the ordeal from yesterday I started out the day with a fresh mind looking for a fresh approach to the task at hand. After some digging around the web I found that there ways in which we can trigger certain scripts in Linux when links are up. The 3g dongle creates a ppp link. All that is needed to place the script after giving it proper permissions at a particular place under if-up.d/ directory of the kind of protocol used in the link (in our case it is ppp). So I did and the results are positive but not consistently. As it turned out my script had some bugs still. I fixed them, it took me the whole day and also logged the output into a file.

Automatic connection on Bootup with Pi

One of the things that make Raspberry Pi so popular is the fact that it is almost a computer that can do almost anything a normal computer can do for a person but at a very low energy consumption. This means that one can have Pi running 24x7 and not be affected by how much it would cost. If a pi is running non stop it only makes sense that it is at some remote place and as such it must be connected to the Internet otherwise how else will it be able to communicate with any other computer. So one of the important things is to make your pi connect to the internet automatically when it boots up. This is useful when for some reasons your pi has to be rebooted and you want it to become active on the network without you being there. This is exactly what I wished to do with a pi today.

The internet connection was to be made through a 3g dongle and using the sakis3g software. In order to connected to the dongle, a single command is to be run. This makes the whole thing very easy. All that is needed is to make sure that this command is run every time pi boots up.

One of easiest ways to do this is to have that command written inside the /etc/rc.local script. So I did and it worked.

Checking availability of BLE in Linux

Most modern PCs have Bluetooth as part of their hardware. But not all of them are Bluetooth Low Energy enabled. I am going to describe a quick way to determine whether your Linux machine is BLE enabled or not. 

First thing to check is to see if the machine is Bluetooth enabled. To do so, follow this link.

After confirming that Bluetooth is there and is running, you are ready to enter into the world of BLE provided of course you're able to.

Open up a terminal and type the following command, assuming hci0 is your adaptor:

$ hciconfig hci0 lestates

This command will return one of two kind of output depending on whether BLE is present or not. If you get something like this:

Read LE supported states on hci0 returned status 1

Then this means that BLE is not present and this is not a limitation you can fix by tweaking software. You need to purchase a Bluetooth Low Energy adaptor and plug it in to your machine. 

On the spin side, if you see a list of states offered by your machine then that obviously mean BLE is good to go. The output may look something like:

Supported link layer states:

YES Non-connectable Advertising State

YES Scannable Advertising State

YES Connectable Advertising State

YES Directed Advertising State

YES Passive Scanning State

YES Active Scanning State

YES Initiating State/Connection State in Master Role

YES Connection State in the Slave Role

YES Non-connectable Advertising State and Passive Scanning State combination

YES Scannable Advertising State and Passive Scanning State combination

YES Connectable Advertising State and Passive Scanning State combination

YES Directed Advertising State and Passive Scanning State combination

YES Non-connectable Advertising State and Active Scanning State combination

YES Scannable Advertising State and Active Scanning State combination

YES Connectable Advertising State and Active Scanning State combination

YES Directed Advertising State and Active Scanning State combination

YES Non-connectable Advertising State and Initiating State combination

YES Scannable Advertising State and Initiating State combination

YES Non-connectable Advertising State and Master Role combination

YES Scannable Advertising State and Master Role combination

YES Non-connectable Advertising State and Slave Role combination

YES Scannable Advertising State and Slave Role combination

YES Passive Scanning State and Initiating State combination

YES Active Scanning State and Initiating State combination

YES Passive Scanning State and Master Role combination

YES Active Scanning State and Master Role combination

YES Passive Scanning State and Slave Role combination

YES Active Scanning State and Slave Role combination

YES Initiating State and Master Role combination/Master Role and Master Role combination

Receiving files from a phone using Bluetooth on a Linux machine

Following the other posts in this genre, I am going to describe a way to receive files sent from a mobile phone to a Linux machine using Bluetooth and as advertised, everything is going to be directly from the terminal.

First of all you need to look for a Bluetooth service running on the Linux machine. To do so, list the services out using the following command:

$ sudo sdptool browse local | grep "Service Name"

This will list out all the Bluetooth services offered presently by the machine. In order to receive files we need the service: "Object Push Profile". If it is listed then all is set otherwise you need to install it by either following this tutorial or simply following these commands:

$ sudo apt-get install obexpushd

Check if the obex-data process is still running:

$ ps auwx | grep obex-data

This will return a process id if the process is running otherwise nothing will be returned. Assuming the process id is XXXX, apply:

$ kill -9 XXXX

Start the service now using the following command:

$ sudo obexpushd -B -n

It should give the following output:

This software comes with ABSOLUTELY NO WARRANTY.

        This is free software, and you are welcome to redistribute it

        under certain conditions.

        Listening on bluetooth/[00:00:00:00:00:00]:9

        

Keep this thing running and open another terminal. Following the procedure given in this post, connect with the phone you wish to communicate with.

Now it is time to send files from your phone. Your machine will be ready to receive the files by now. When the transfer is done, you can quit the interactive session and kill the obexpushd process by doing ctrl + c.

Sending files using Bluetooth from terminal to an Android phone

In the last post I described how to get Bluetooth up and running on a Linux machine all from the terminal. In this post I am going to describe one half of the most obvious use case of Bluetooth i.e. sending files from the Linux machine to another Bluetooth device e.g. a phone.

I am assuming that you have a Bluetooth adaptor running, if no then check the above mentioned post on how to do that. 

First thing you are going to do has little to do with your Linux machine. You need to make your phone a ftp server. Don't worry, it's not very fancy. An app installation will do just fine. The app I'd recommend for this is the Astro Bluetooth Module by Metago. Once the app is installed, open it and tick the following three things (you are gonna wanna do this every time you need to receive files):

• Bluetooth
• Discover-able
• OBEX ftp server

OBEX is the Object Exchange profile that is designed specially for the transfer of files between two Bluetooth devices.

Now it is time to turn to the Linux machine and make sure it is ftp ready. First thing is to just update stuff just for good measure:

$ sudo apt-get update

Then you can go ahead and add obexftp:

$ sudo apt-get install obexftp

Now it is time to hunt for the phone you want to interact with. Use the following command to scan all nearby Bluetooth devices:

$ hcitool scan

It will start scanning like so:

Scanning ...

B4:CE:F6:B5:45:D7 Nexus 9

D8:5D:E2:A8:0B:B4 DIPTENDU-PC

5C:E0:C5:56:D3:CF PRIYARAN-E7450

You need to copy or at least make a note of the address of the device you wish to communicate with. Lets assume the address of the device is B4:CE:F6:B5:45:D7.

After this you're going to use an interactive tool provided to you already in order to do any interactions using Bluetooth. The tool is bluetoothctl and you can invoke it using the command:

$ bluetoothctl

Trust me and type the following four commands directly into the interactive session:

[bluetooth]# power on

[bluetooth]# agent on

[bluetooth]# default-agent

[bluetooth]# scan on

Now connect to the device using its MAC address:

[bluetooth]# pair B4:CE:F6:B5:45:D7

[bluetooth]# trust B4:CE:F6:B5:45:D7

Obviously you'll get some kind of verification code on your phone to confirm this connection, accept the invitation. You have now paired with the phone. Without closing this terminal, open up another one. And type the following command:

$ obexftp -b B4:CE:F6:B5:45:D7 -p /path/to/file/to/be/send

From here you should get a prompt at your phone showing the transfer happening.

Once the file transfer is done, exit the terminal window and quite the interactive session by typing:

[bluetooth]# quit

To know more about these interactive commands, you can use 

[bluetooth]# help

Finding, enabling and starting Bluetooth in Linux from terminal

First thing that is needed to be done is to check if your machine has a Bluetooth adaptor or not. Most modern PC should have one builtin but if that is not the case one can always buy a cheap Bluetooth USB dongle and plug it in. Once the hardware side of things are done with, enter the following command to start the essential software routines to handle Bluetooth:

$ sudo service bluetooth start

To know what and how many adaptors you have at your disposal you need to use the following command:

$ hciconfig

The above command will return nothing if no Bluetooth adaptor is available, else you will see something like this as output:

hci0: Type: BR/EDR  Bus: UART

BD Address: B8:27:EB:E4:87:C3  ACL MTU: 1021:8  SCO MTU: 64:1

UP RUNNING

RX bytes:773 acl:0 sco:0 events:50 errors:0

TX bytes:2540 acl:0 sco:0 commands:50 errors:0

What you see above is the primary information about the adaptor hci0. The zero here is like a serial number, if the system had two adaptors then there would have been two such outputs following hci0 and hci1. For the rest of the commands in this post I am going to use hci0 as a default.

If you wish to know more about the adaptor then use:

$ hciconfig -a hci0

In the output of the above commands you see in the second line something written as "UP RUNNING". This represents the current state of the adaptor. Currently hci0 is running. To shut the radio down use:

$ hciconfig hci0 down

To bring it back again:

$ hciconfig hci0 up

That is it. Bluetooth is now functional. As a validation you can scan using other devices to see if your device gets listed there. It should.

Once you're able to get this and are willing to continue using Bluetooth from the terminal, you need to install the following things for a better experience:

$ sudo apt-get install bluez

$ sudo apt-get install bluez-tools

Sakis3g connection configuration

Sakis3g is a software that is used to connect to 3g modems of various types. But as is the case with most things in Linux, there is some detailed amount of configuration required for this to work. And of course these is no concrete documentation except for the Internet to go and figure this out. But fortunately there was one link given to me that I guess worked for some other employee. So I had hope. So I followed the instructions given in that link and was able to connect the Rasberry Pi to a 3g airtel dongle using sakis3g.

The next step was to be able to this automatically on bootup. 

BKL: One lock to bind them all

Synchronised access by multiple resources is an ability any decent OS kernel must posses. The access is not intended for just the user. In fact the kernel internals are the largest consumers of synchronised access.

The kernel has to micro manage the whole hardware of the machine as such you'd expect it to be intensely intertwined and cutting corners in terms of performance at all stages. Even if performance and efficiency are very much desired, one thing that cannot be compromised is the security and safety of it all. A common backdoor in the safe working of the kernel is often a vulnerability of the concurrency of its functions. So before one considers being clever in infinitely many ways for the purposes of improving the kernel's performance, one must be singularly confidant that the kernel is safely concurrent. A kernel not using the hardware to its full potential is ordinary but a kernel risking reliability by not preventing the hardware from over stepping processes is criminal. It goes without saying that locks are absolutely essential in a kernel.

I know almost nothing about kernels. Thanks to the Linux kernel, that almost wasn't an absolutely. Not long before the present versions of Linux, the famous kernel had an old companion called the big kernel lock. It belongs to the spin lock category. The reason for the use of the article "the" in its name is because of its exclusiveness. It was about the only lock the Linux kernel employed or needed to hold its own at concurrency of operations. The adjective "big" states the obvious that to manage a big piece of software, a big lock is needed. The bigness of a lock in my guess is estimated by the size of the critical section it holds, hence the name the big kernel lock is justified. 

The big kernel lock was a global spin lock responsible for most synchronisation efforts in the Linux kernel back when it was active and acting. Remember what a lock is? If no then this is awkward otherwise following are some interesting features of the big kernel lock:

1. A process can sleep and probably dream while holding the BKL. This means that a process doesn't have to relinquish the lock (which it probably should), but the lock will be released automatically anticipating the process' inactivity.
2. The lock is recursive in nature meaning that a single process can have consecutive and successful attempts at locking the kernel (or at least a big significant chunk of it).
3. Like the ring in the Lord of the rings, you don't want this as it corrupts, eventually.

There has been radical changes in the Linux kernel to remove the lock and although it hasn't completely been removed but its usage has been reduced dramatically after 2011. New code is discouraged from using the now infamous lock_kernel() API.

The BKL does not allow kernel preemption. This is a good thing when it is a good thing. I mean BKL is essential for cases where it remains used in the kernel still but it cannot be used extensively for locking the kernel. Doing so would be like jamming the whole city's traffic even if a small gathering was to happen in a corner of the town. A more fine graded locking mechanism is apt for such purposes. Such fine grading has been gradually applied in the kernel since. BKL is obsolete but it isn't extinct. For more informed and authentic information, read this link.

Locks and Types

Locking is essential for computers for the same reasons door knobs are essential for bathrooms. A lock is meant to ensure only one (or any other number) of contenders access a resource. By access I meant exploit and by contenders I meant entities that need the resource ideally immediately.

I mentioned how the number of entities accessing a lock can be more than one. This is because a resource may have more than one available copy of it self for the greater good of the consumer kind. The multiplicity of same resources is there for the same reasons that explain multiple compartments in a public toilet. In short it is good to build more hoping that no one has to wait much and hence no one has to explode. I was talking about computers if the above metaphors got mixed a little to homogeneously.

The basic idea is that a lock protects access to some kind of shared resource. If someone has the lock, that someone can use the resource. If someone does not have the lock, they have to wait. In order to "have the lock", one needs to wait for their turn in perhaps a queue or a skewed web of priorities (just like life is). Once the turn comes, the following abstract sequence follows:

1. Acquire the lock on the resource.
2. Do your thing, quickly but to your satisfaction.
3. Hope no one important comes and steals your turn in between "the act".
4. Release the lock when done, or release it anyway and wait till your turn comes back again.

The concept of locks is not just to let someone have protected access to a resource but is also to keep others from rushing into a busy process. Such waiting folks are said to have been blocked, temporarily.

Now moving on from what are locks to how they are implemented. Implementing a lock is as simple as counting stuff. You just need to count how many resources are available and how many are free. A lockable resource will have a maximum number of owners or lock holders. If that number has been reached then no one will be allowed to use it further simply because no one can. If and when a resource frees up, the next in line is allowed to access it. Simple. 

Enough for the overly simplified analogies for locks. Let us look at some types of locks:

1. Mutexes: Mutex is short for Mutual Exclusion. The word exclusion is anti to the whole idea of sharing. So mutex are locks that allow exclusive access to a resource to only one process at a time. Unless the degree of mutexness is diluted by excuses like shared mutex, recursive mutex or read/write mutex; a mutex most certainly means no sharing at all.
2. Recursice Mutex: It is the same as a regular mutex but more than one lock can be held on the resource at the same time. So they are nothing like mutex at all. No, but seriously they are, maybe not by intent but by principle. These allow the same process to hold multiple locks over the same resource, recursively. The complete freedom of the resource would mean an completely inverse unwinding of these locks. 
3. Reader/Writer Mutexes: These mutexes add more meaning to the notion of holding a lock over a resource. They classify locks to be of two kinds:
   1. Read locks: Locks given to threads that only wish to read the resource.
   2. Write locks: Locks given to threads that wish to make changes to the resource.
      If some process has a write lock at a resource then no one else is allowed to either change the resource or even read it. But if some process is having a read lock over the resource then other processes can also have a sneak peak and acquire read locks of their own.
4. Spinlocks: A spinlock is a special kind of mutex. Why? Probably since it does not need to be appended by the word mutex to have some respectable identity. In fact there was a spinlock in the Linux kernel that was so fundamental and so fundamentally wrong that they actually had to remove it or Linux would not have been so sophisticated.
   A spinlock does not use OS synchronisation functions when a contending process has to wait for its turn. Instead, it just keeps trying to update the mutex data structure to take the lock in a loop. It is the relentless type. This type of technique resonates with the inpatient types and hence works well when the locks are generally held for a short time and are changed hands frequently. However if some lock is in place and is meant to stay in place for a long time then such consistent looping to get the lock may seem and is a waste of time for the processor.
5. Semaphores: Semaphores are like mutex made rich and diplomatic. They have the powers of a mutex minus the rudeness. They deliberately interact with multiple processes and multiple resources. With each resource there is a number representing how many processes are currently consuming the resource and how many are currently craving it. I have just the post for you if you wish to read more about them. Read more about them. 

Opaque data types

Data types are tools to describe data. But to whom is this description meant for? One would assume that since a programmer is writing the data type, it is meant for the programmer. But there are cases when the data type written by one programmer is made available to another without the source or any knowledge of what the data type contains. What the later is provided with are some guidelines of how and when to use the instances of that data type. The only thing allowed is for someone to pass the objects of the data type around without ever touching the internals of it. Such data types are called opaque for literal reasons.

One example of opaque data types that is given in abundance is that of the FILE structure in C. The argument goes that since we never do things like:

FILE *obj;

obj.some_member = some_value;

We can say that this is opaque. This is true by definition if no one is made aware of the structure of this struct. And if we are not made aware of the structure then we can't access the members, simply because we won't know what the members are. All we can do is use the objects or pointers to the objects and pass them around to functions that know how to manipulate these. Examples of such functions would be fopen and fclose in the C language that are used to work with FILE structure pointers.

This implies that since such functions can manipulate these structures, they must know what the structures contains and hence these functions must also be made opaque from the user programmer otherwise he/ she could just look these functions up and code a similar one based on the knowledge of the internals of the structure. This is not like the creators of such types don't want us to see what they have done. It is because we don't want to know what is inside. Each data type is there for a purpose and the purpose of opaque data types is realised completely with the APIs provided to use them. Extending its internals to the programmer will do no good and may even prove to be counter productive.

Think of the opaque data types as black boxes that are too important for one to open up and play around with. They are meant to be played with as is i.e. like a black box. It is not as if the internals of such types are an absolute secret. It depends entirely on the people implementing the library that is providing these. Some chose to provide the source, others just offer the binary. Any which way, the internals of the structure are not intended for use in a program hence they shouldn't matter whether they are available or not.