Ethics and AI: How Human flaws translate to Artificial Intelligence?

“Two things are infinite: the universe and human stupidity; and I’m not sure about the universe!”

Humans are perfect and human behavior is impeccable. This claim is not true by any stretch of the imagination. With a population bulging over 7 billion, history of wars, political and geographical differences of varying proportions, virtually uncountable religious and cultural beliefs and the inherent complexity of the human psyche; it is safe to say that human behavior is equally difficult to model as it is far from the ideal. Almost any action being undertaken by man throughout the globe has an inevitable uncertainty of outcome as humans are bound to make mistakes. The fundamental purpose of machine automation is to eliminate this very inconsistency and inefficiency associated with humans. Dealing with things like inconsistency and inefficiency is easy with machines that function in closed environments. But there is a whole other aspect to human limitations. There are decisions that humans can’t make; not because of our biological shortcomings but rather due to the sheer scope of the implications of such decisions. And such decisions are not obscure at all. We face many situations in our day to day life where our actions (or lack of any) can potentially lead to serious consequences but we continue to be arbitrarily irresponsible out of moral and/or mechanical incompetence. This, unfortunately, is the accepted state of mankind but what worries me a lot is the continuous process of giving control over to autonomous machines for such delicate and potentially disastrous scenarios.

A common discussion in most AI ethics workshops (including this one) is around the hypothetical and menacing situation of a train on course to kill either a group of 5 people or a single person and the decision to choose between 1 death and 5 is left to the audience by virtue of a button which changes the train’s course. There are other variations that offer a choice between the life of a toddler and an old man. All these scenarios offer the same fundamental question of can we quantify or evaluate the life or death of anyone like that? But this question is still just on the surface. The much more important and hidden question is should we quantify or evaluate human life or death? Sure, to some people (including myself), letting 5 people or a toddler die is much worse than letting a single person or an old man die. But a single person does not have the right and therefore, should not have the power to make such a decision. This example is popular because it leads directly into the realm of self-driving cars. I have realized that designing a self-driving car, for example, is not just a matter of building a system intelligent enough to handle all the physical and mechanical intricacies of the road. It must also include ways to handle misfortunate situations where human lives are at risk. And that is where I feel there is no right approach. A human shouldn’t decide which person to kill (by letting the car drive over in case of an accident) and so shouldn’t any AI system created by humans.

Another discussion covered in the workshop was about the various types of bias induced in autonomous and intelligent software. The basis of many of these biases was rooted in human discrimination. As mentioned previously, our perception of the world is far from ideal. There is a tremendous imbalance of power and influence in human society. Power corrupts and absolute power corrupts absolutely. People, corporations and even governments do not always act for the common good. Their actions are sometimes motivated by greed (wrong use of data collection), misguided by bias (systems used by police) or ignorant of the long-term effects (a unanimous shift towards autonomous weapons). And unfortunately, the fate of the majority of AI is in their hands. So much of software development is incremental and if corporations or governments continue to churn out software that is not transparent and fair this can lead to software gaining not only in intelligence and influence but also in secrecy and malintent. This is a dangerous cycle that is only gaining momentum and it needs to be corrected.

Mimicking human judgment is hard enough. It may even be impossible. But one thing is for sure, there are situations where even humans can’t comprehend all implications of an action and as such, we cannot expect any AI in the same situation to do the “right thing”. In these impossible situations, both humans and AI will be out of options but with a more potent and ruthless AI, the scale of destruction will be much greater. Therefore, it is important to not give absolute power to AI in such cases. And then there are situations where I feel it is imperative to separate AI from human bias, greed, and corruption. History suggests that we are excellent in self-inflicted damage. So, as mankind is marching forward in the creation of things that are potentially more capable than humans then we must be very careful that we do not end up with things that do more damage than good.

Ethics and AI: The Charade of Privacy

It is very difficult to put a definitive status on privacy in today’s digital age. I am not sure if privacy is completely dead or if it will die in a certain period of time. But one thing I am quite confident in is that privacy is dying and it is dying fast. The diminishing privacy is not a consequence of a single phenomenon. It is in fact, caused by the combination of a variety of things. Ever increasing volume of data being generated, computer algorithms becoming abundantly sophisticated and humans growing in eagerness to share every aspect of their lives with the world; these to me are the primary factors that are proving to be fatal to the concept of privacy as we know it.

The very first thing we need to consider is the near unfathomable size of data being produced by a single human on average. As per the statistic given in the key note speech at 2017 CeBIT Global Conference by Dr. Michal Kosinski, a single human produced 500 MB of data per day. This statistic is over two years old and therefore this number must have increased by factors already. Not all of this data is produced intentionally by humans. In fact, most of it is gathered silently by companies governing the Internet. “If you are not paying for a service, you are the product.” This quote sums up the situation perfectly. If companies on the Internet are incentivizing the end of human privacy then us humans are sponsoring it by giving away the perfect resource to extract out every little detail there is to extract from our lives. For me, the problem is not that data is being collected in various manners from our digital footprint, but it is the fact that we are so careless in our choices when it comes to data and so many of us are keen on voluntarily broadcasting their lives to the world. We are actively contributing to this ocean of data that is basically an essence of our online existence. For example, it is one thing that our location data is being recorded by Google, but what can be made of the pictures we share on Instagram, blatantly showing off where we are and what we are doing. Companies and the government are data hungry and humans are so full of themselves that they can’t stand the thought of being unnoticed. And this creates a recipe of disaster for anonymity on the Internet.

Artificial Intelligence is burgeoning and algorithms are becoming smarter, quicker and more efficient in the way in which they make use of the information available to them. Algorithms today don’t need a complete and connected dataset pertaining to a person in order piece together a decent analysis. Even if there are attempts of scrambling sensitive information in order to lose meaning in the shuffle, computers are pretty good at recognizing even the feeblest of patterns and eventually singling out humans from these patterns. One example of how efficient machine learning techniques are at extrapolating personality trait from a very basic set of data is given in a study that used Facebook likes of few million people and predicted things like sexual orientation, political views and general personality. The study showed that with a few hundred likes, the algorithm does a better job at predicting a person’s behavior than all of that person’s friends, family and spouse. It is established that keeping secrets from family and spouse is never easy, then how can a person’s privacy be kept from such algorithms that know more with so little amounts of data? The simple answer is that it is impossible to maintain privacy specially with amount of access we have allowed in our lives. The fact of the matter is that cracking privacy is like finding patterns and computers are pretty good at cracking patterns specially when we are not making it hard for computers by providing as much data as they like.

Attempting to protect privacy in my opinion is a lost cause. All the companies that employ user data need to be regulated. But how can we have hope when the regulator i.e. the government itself is big on gathering data from people and using it to control us.  To be honest, I don’t see much harm in all of this. I am not complaining about the lack of privacy, I am just stating its inevitability. For me, privacy is relevant with respect to other people. As long data is being kept from people in a person’s social circle, it is fine if some corporation uses it further their business. Afterall, the overall goal is to better server the consumer and if that’s the case then I see no problems with our behavior being monitored. Instead of making futile efforts to stop being controlled and monitored, we should make efforts to ensure that we are being controlled and monitored for the right reasons.

Ethics and AI: Is it acceptable to exploit AI or robots and force them into servitude?

I believe it is acceptable and even desirable to use AI or robots in ways and magnitude that fringe into servitude. I do not agree with the notion of comparing using an inanimate technology to exploitation. I also don’t think AI and robots are being forced into serving humans because as of now, they do not possess any other desires that can be forcibly suppressed in favour of servitude for humans.

Although we fear the lack of human contact that could result from being surrounded by robots, for some tasks, many would prefer machines rather than humans. This to me captures the purpose and motivation behind having AI or robots perfectly. AI was never imagined to be something that should enjoy the same fundamental rights as the living beings on this planet do. It was conceptualized as a tool to do things for us in an efficient and intelligent manner. Any AI technology should be treated like the resource it is and not as a mortal being. And as is the case with the consumption of any resource, we should be responsible and careful so as to not run it into scarcity or extinction. For me, the question is not whether AI or robots are being forcibly exploited into our service but it is whether or not we are enjoying the services of these ever-improving tools sustainably.

The important thing to always keep in mind when thinking about AI is that it is fundamentally different from any form of life on the planet. There may be increasing efforts in making robots like humans but it is hard to see how these robots can be induced with human consciousness. There has been significant support for the argument that consciousness can only arise and exist in biological matter. And therefore, it is hard to imagine AI robots having human emotions. My point is that an entity being forced into servitude is dangerous if it is capable of damage and has the ability to feel vengeful or angry. While there is no denying the fact that AI can very well inflict great damage to mankind as well as this planet but I doubt that it will do it because of how it was treated unfairly by its “masters”. If the act of using AI relentlessly does not by itself warrant any retaliation then I don’t see any reason to act restrained in this regard.

Singularity or The Intelligence Explosion are popular notions in both the world of scientific research and pop-culture. There is a myriad of arguments that argue against the possibility of a singularity like The Anthropocentric Argument (surpassing human intellect is not some milestone beyond which anything is possible), The Metaintelligence Argument (emphasizing that the capability of doing a task well does not necessarily lead to an improvement in this capability) and The Limits of Intelligence Argument (questioning if there is a limit to intelligence in any form). It is unclear or even doubtful that singularity is a possibility and even if it is possible, there are arguments that it will not directly and immediately imply catastrophe. But even if we assume what science fiction has always offered to be true and believe that singularity means disaster for/ extinction of the human race, then doesn’t it mean that our future is sealed regardless of how we treat AI now? What difference will our current behavior make to a dystopian future where AI will deem humans as a non-necessity and wipe us from the face of the earth?

In conclusion, I think AI and robots are built to serve humans and there is nothing wrong with being strict in our use of AI as long as we are responsible for the sustainable consumption of this and other resources on our planet. There are many considerations before assuming that AI will eventually become supreme and render our species obsolete. But even if such assumptions are to be made out of extreme caution, I think such destruction is contingent upon how we develop artificial intelligence or how we use it and not how severely we force it into servitude.

Purging Outlook Account

There are two things I don't like about Microsoft Outlook. First is logging in to it and second is logging out. Log in is just a time consuming process that irritates for sure but does bring results later rather than sooner.

But logging out is a different ball game. Not only is your calmness challenged by the astoundingly long process of actually finding a sign out button, your hopes are crushed too by the inability of that button to actually sign you out. Nothing works, nothing ever works.

But after struggling for many crucial minutes I finally found a deadly looking solution that worked for me. It worked wonders. It worked magically. It signed me out of Outlook.

Here it is, put it in a frame if you must, I am putting it in my beloved blog:

Control Panel --> Mail(32 bit) --> Show Profiles --> Delete all profiles.

Structure Packing

Run the following piece of C++ code and observe the output:

#include <iostream>

struct something {

        char c;

        int i;

        short s;

};

int main()

{

        std::cout << sizeof(something) << endl;

}

The output will be 12.

Now run the following program:

#include <iostream>

struct something {

        int i;

        short s;

        char c;

};

int main()

{

        std::cout << sizeof(something) << endl;

}

The output will be 8.

Now that you have run both versions of the same program on a computer, run them in your head. Both programs are doing the same thing. They are printing the size of a struct. The struct contains the same type of variable in both cases. And yet the size of the struct comes out to be different. The structs are almost identical with the only difference being the order in which the variables are declared inside the struct. So the moral of the story is that the order in which variables are declared in a struct determine what the size of struct is going to be. But you should ask your self that why in God's good name should the declaration order matter to size, after all the same number and type of variables are used and hence the size of memory they occupy should also be the same. Well clearly this is not the case. To understand why, you need to learn this first. 

Data alignment! It was fun wasn't it? A direct consequence of this sort of alignment of data in C (and C++) is the way in which compound data types like structs are given memory. Remember how C likes its variables to be stored at memory addresses that are a multiple of their size? Well C doesn't discriminate, the same rule applies to structs. So if a struct is of size S, then any instance of this struct will be stored at an address that is a multiple of S. This is very intuitive as you already know why this is required. The issue is the fact that since a struct is a compound data type, its size can be any arbitrary number of bytes depending on what variables it holds as members. The whole alignment thing works only because the data types were allowed to have certain convenient sizes i.e. 2, 4, 8 ... This means that if structs are allowed to have any arbitrary size then aligning them according to their size will make arbitrary sense (i.e. it will work well sometimes and sometimes it will seem stupid).

Hence it is imperative that the compiler do something to make sure that the structs also end up getting convenient sizes i.e. multiples of 4. The rule that C has to ensure this is rather simple, The size of the struct is always made to be a multiple of the size of the biggest member of the struct. In the example above, the struct something has int i as its biggest member with size 4, so the struct size will also be a multiple of that. How does the compiler achieve this? By using structure packing.

To understand this, you need to be introduced with some nomenclature. X from now on is the size of the biggest member of the struct. Boundary refers to any multiple of X following the address where the struct memory begins. So some struct object begins its memory at 20 and X is 8 for that struct, then the boundaries are: 28, 36, 44, and so on.

Structure packing is a technique (or a policy) in which if adding a new member of the structure to memory crosses any boundary in the struct, then before adding that member, the leading (proceeding) bits are left as padding and the new member is made to start immediately after the said boundary ends. Finally if after allocating memory to all members in this way, the struct is shy of the next boundary by a few bytes, then those bytes are also used as padding just to make the structure a direct multiple of X and end always end up on the boundary exactly.

Consider the first example of the struct something. The value of X is 4 (size of int i, the biggest member of the struct). Adding the first member of the struct, char c crosses no boundaries and the size of struct reaches 1 (which is the size of char). Then comes the turn of int i. int i is 4 bytes in size, adding it to the memory of struct will make the sum 5. This is problematic, int i is crossing the boundary of the first multiple of 4 i.e. 4. In this case the compiler will decide to allocate 4 bytes to int i starting from byte number 5 only and the bytes in between (2, 3, 4) will be used as padding. So int i will start from 5 and will go till 8. The variable left to be added now is short s which is 2 bytes in size. Adding it to 8 does not cross any boundaries so no proceeding padding is needed. But since the final outcome comes at the total of 10, an extra two bytes are padded just to make the size be 12 i.e. a multiple of 4. This way the final size comes out to be 12 bytes, out which 7 bytes are the variables and 5 bytes are just padding. That is some extravagant wastage don't you think?

Now moving onto the second example. The value of X is still 4. The first variable to be added is int i, adding it gets the total to 4. This covers the first boundary entirely. Then comes short s. Adding this takes the total to 6. So far no addition has cost the total to cross any boundaries. Lastly it is required to add char c. c is 1 byte in size. Adding it to the struct makes the total memory equal to 7, which as you may notice still does not cross the next multiple of 4 i.e. 8. Hence so far we are golden, no padding is needed. The only thing left to do is to add one byte of padding in the end to take the total to an even 8. And vola, that is the final size of this struct, with 7 bytes of actual data and a frugal byte for padding.

You see now, how rearranging members can make a difference in size of the struct. One might say that okay saving a few bytes is cool but it isn't very significant. That may be true for moderate size programs on moderate size computers, but if either the program is very big or the computer on which it is to be run is low on memory, then these little alterations can add up to amazing benefits.

The next time you are writing a struct, may be keep in mind to arrange the members in such a way that the small ones come together so that they can be fit inside a single chunk and padding is needed to the least.

Another important thing to know here is that the standard does not guarantee that the padded bytes are to be zeroed, they can be anything, from zero to any arbitrary value. It is the programmer's duty to play around with these carefully. Assuming that the members in a struct are all packed together continuously is plain foolish.

Data allignment in C

I am fan of travelling. But just not as a hobby or an activity I'd like to take, instead as an analogy to understand and hence communicate some efficiency concerns in computers. I am going to do this again.

When one has to travel and that to frequently then what you'd expect the number of items of their luggage to be? As less as possible right? A traveller (at least a wise one) would want to pack things in (say) one bag so that carrying stuff isn't as prominent a pain as it otherwise would be. Now assume that on a single trip, one is allowed to take only one bag. Considering the trip costs money and more importantly, time, one would want to take this trip once only. Packing your stuff in two separate bags would be stupid as then you'd have to travel twice in order to complete the transfer.

The idea is to keep things packed in such a manner that one would not require more bags to carry them. You want only one bag or at the most, a number of bags that you can carry at once.

A program travels too, more frequently then any human in any imaginable circumstance. And while travelling, a program also has to carry stuff from memory to the CPU, or back. As such you'd want your program to carry the whole data belonging to the same variable once, in one data cycle. A 64 bit CPU can carry 64 bits at once. So if you have 65 bits of data in memory then that will force your CPU to take two trips in order to fetch this data entirely. But this is okay as no altercation would make this transaction more efficient, CPU can only carry 64 bits in parallel.
Another interesting peculiar feature of CPUs is their habit of considering only those chunks of memory that start from an address which is a multiple of the word size. So if the word size was X then the CPU would only fetch memory chunks starting from 0, X, 2X, 3X, ... This means that if some thing is placed in such a way in memory that it is not wholly a part of any of these chunks, then the CPU will have to make two trips in order to fetch it and then will have to perform some bit shifting in order to restore the proper value of the thing being fetched as well.

Data types in C (with exception of char) have even bit sizes. Their size is always a multiple of 2. More importantly, except for chars, all other primitive data types in C are either a multiple or a factor of the word size on most architectures. Most modern architectures have a word size which is a multiple of 4, so this means that almost all primitive data types in C should also be a multiple or a factor of 4 bytes in size. You won't see a data type in C that is primitive and has a size of 6 bytes or 10 bytes or 14 bytes with the exception of long double. The size will be either 2, 4 or any multiple of 4.
When a compiler has to store a variable of a particular type, the starting address available to store the variable can be anything (depending what and how things were packed before). If the compiler just went ahead and stored the variable exactly at the address that was available then that could lead to some inefficiencies in getting that variable back from memory. For example consider that the variable size is 4 and the starting address available to store the variable is 18, then if the storage was done simply, the variable will reside in memory from the address 18 to 22. This will lead to the same problem described in the earlier paragraphs. Since the variable will be residing in two separate memory chunks (that start from a multiple of 4 and are 4 bits in size), the CPU will have to take two trips in order to fetch it completely and then will have to perform some bit shifting on the partial values of both the chunks in order to retrieve the proper value of the variable. Now ask yourself, wouldn't it be nicer if this variable was stored at address 20 to 24 even if that meant leaving two empty bits at 18 and 19? Of course it would, as now the CPU could simple fetch the variable in one go and would not require any manipulation of bits in order to get the proper value of the variable.
For this reason the C language enforces some rules that determine how the variables of various sizes are to be stored. The general rule is that if a variable is of size S then it must be stored at an address that is a multiple of S as well. Since the the size of variables (as mentioned above) align (mostly) with the word size of machines, this enforcement ensures that no variable has to be split between two memory words.

Spartan Programming

You all have seen the movie 300 which makes you enough of an expert to understand this post. It would help if you knew some programming as well but it's just an after thought so don't beat yourself for it.

Remember how Spartans are almost anti-fancy in their attire and yet a menacing force of nature? Well it so happens that there is a culture in programming inspired almost from this principle i.e. be as effective as possible without using too many tools. This culture is aptly named Spartan programming.

A programming work can be deemed as being Spartan based on the following criteria:

1. Horizontal complexity: A program shouldn't be too nested. More nesting there is, the more chances you have of clearing your logic and making it more streamlined.
2. Vertical complexity: The length of the program should be as short as possible. No point in writing more code when the problem can be solved with something shorter and hence sweeter.
3. Token count: A token is any entity in a program that have individual meaning be it your variables, literals, keywords or anything that is allowed by the language and is part of your program. It is obvious that reducing token count without loosing meaning can ensure simplicity to your algorithm and hence can be more effective and scale-able.
4. Character count: Don't be too verbose with your comments and variable names. Be brief and to the point.
5. Parameters: It is always nice to refactor a solution into subroutines. A good subroutine should have a clear purpose and should be spartan in itself. One of the essential elements of ensuring this would be reducing the passing of implicit data to subroutines as arguments. You don't want to pass things that are available otherwise and can be derived from other parameters.
6. Variables: Don't overuse a variable in too many places. It is also considered a good practice to declare variables immediately before they are to be used and not all together at the top of your program.

In conclusion, spartan programming encourages frugal use of resources as a programmer. Another important feature is to make your solution robust just like a spartan. A spartan isn't invincible but it certainly should be tough enough.

Programming is about dumbing things down.

The more code you write, the more you wish you had written less.

What I mean by this is that after completing almost any programming assignment, one looks at ways in which the code could be optimized. Since optimization is a much more intellectual task, programmers like me try to confuse it with the size of code. I belong to a category of coders that think less the code more optimized the algorithm is. And you know what, for once I think I am on the right side of things in computing.

While being able to write crazy amount of code is a fantasy we all have but no one would question the beauty of an elegant solution to a tricky problem that saved day at the office. Computer scientists make a living out of telling dumb things (computers) to do dumb things (instructions) in order to achieve impressive feats (solve a Rubik's cube or simulate the universe, take a pick). What this process does is that it eventually breaks in a computer scientist the egoistic complexity a human has innately. They (I hesitate to include myself) tend to think about things from a more elementary point of view. Redefining problems in order to make the solution seem much more obvious,  classifying behavior in order to be able to compute it and being succinct in what they have to offer for the sake of being fast and not more omniscience.

An awesome algorithm may not be one that solves a hundred problems at once but rather an algorithm that solves a single problem simply. It is important for a coder to realize that computers can never be as smart as we are. Getting smart requires evolution and evolution takes time. So in order to be effective as a programmer one needs to learn to solve problems from a much more dumber perspective. Once we move being intelligent out of the equation we can start focusing on being less complicated. And once we are less complicated, we are less inclined to be wrong. And what do we call something that is not wrong? Right.

Write programs that can make sense to computers, after all they are the ones that have to run it.

UML: User Mode Linux

Linux is a name for a kernel. A kernel is a big piece of tough software that makes the wilderness of hardware behave for the comfort of the user. The user may run applications of varying tastes on the hardware without ever having to explain the billion gates inside of a computer, which way to let the current follow and to hold on which side. It is all taken care of the kernel.

But what if I told you that the kernel too can be run as an application. After all it is in the end a piece of software although a little firm. This idea isn't that alien of course. It kind of resembles virtualization.

You see, Linux being a kernel is meant to work on and with hardware. But if one wishes to run Linux in user mode i.e. like an application then it can't be given the kind of access it needs to the hardware. So what is the best that can done? Visualize an entire machine of hardware in software (simulate) and allow the application (Linux) to run the courses on it. It is important to note that the host for such activities must also be running on Linux.

It is an immediate given that this is fun. Running multiple copies of a kernel on machine simulates having multiple machines working independently of each other on a single big piece of hardware. Apart from being funny, this can also be very useful. The idea of having a kernel but not with unlimited power and almost no authority over your not so free and hence obviously precious hardware makes perfect sense for innovative development and kernel tweaking. You can use this technique to alter the kernel as much as you like without putting your computer in harm's way.

Even if you have some sincere activities planned, you can seek great aid from such virtualization. For example multiple servers can be hosted from a single machine this way. It can also be a great way to teach students about kernels and Linux.

All things said, this technique is not yet comprehensive. There are many leaps of development needed before it becomes more popular and hence starts gaining some developer attention as well.

Let's ID UUIDs

A UUID stands for Universally Unique Identity. Pretty heavy don't you think? To claim that something is unique in the entire universe is a bold move. Anyway, a UUID is a number representing the type of data stored in BLE servers, beacons and anything that uses Bluetooth and stores some data.

The way Bluetooth technology looks at data is centred around UUIDs. A UUID is used to classify data into certain types. It is just like a data type like your ints and floats of the world. The difference is that each type is a huge number and there is a huge number of these types.

Like in any programming language, same binary digits would mean different things depending on what type of data they are i.e. what is their UUID.

Now that you know what a UUID is, I want you to question who decides these UUIDs? The obvious answer would be the people that came up with the whole idea of UUIDs i.e. the Bluetooth SIG. This answer is true but not complete. As it turns out, there are two types of UUIDs that can be found:

One type is of the UUIDs that are 16 bits long and the other is of the UUIDs that are 16 bytes long. So you can imagine that the first type will contain a much lesser number of UUIDs.

The 16 bit UUIDs are provided by the standard. They were set at the time of inception of this specification or in consecutive years but only the SIG. A list of these UUIDs can be found at the Bluetooth website. Since these are provided by SIG, they remain constant. These are sort of like the built in data types supplied by a programming language.

The 16 byte UUIDs are kind of made up by various organisations and companies around the globe (or dare I say the universe) as they go along exploring newer areas and inventing new kinds of data crunched by Bluetooth devices. Once someone has an idea for a new type of data (a new profile), then they have to register that with the SIG and also provide the specification for it. This type will then be given a 16 byte unique number which will not be used by any other type afterwards. The larger length of this type of UUID make sense because the number of such applicants is enormously big. This whole thing is like making a new class and then turning it into a library for the rest of the world to use. The UUID is just a unique ID for your class/library.

Advertising using a BLE beacon in Linux

Beacons are a great way to send brief but important data to nearby scanning objects. They will be a force to be reckoned with in the future i.e. a world full of Internet of Things. In this post I will present a way in which you can advertise like a beacon from a Linux machine with attached or inbuilt Bluetooth low energy hardware. 

Since one expects beacons to advertise constantly, it is more likely that you'll be applying this concept to less consuming computers like your Raspberry Pi's or BeagleBone Black's. 

Let us get on with some primary installation stuff. Most of these things should already be installed on most systems but run these commands just for the sake of it:

sudo apt-get install libusb-dev 

sudo apt-get install libdbus-1-dev 

sudo apt-get install libglib2.0-dev --fix-missing

sudo apt-get install libudev-dev 

sudo apt-get install libical-dev

sudo apt-get install libreadline-dev

sudo apt-get install xclip

After this, you need to start your Bluetooth adaptor (assuming it is BLE supportive, find all about this here). Now lets get to business.

A beacons advertises. You know that. But the next defining thing about beacons is their inability to connect to any scanning device. A beacon just transmits data, it does not engage in a connection with any other BLE device. So the first thing we need to do is get your adaptor advertising in non-connectable mode:

sudo hciconfig hci0 leadv 3

sudo hciconfig hci0 noscan

The number 3 in the above command represents the non-connectable state. To advertise in connectable state use 0 instead and loose the successive "noscan" command.

Your adaptor should be advertising now. Now it is time to put some data in the advertisement. The data to be advertised has to be classified into a type. The beacon must also tell what type of data it is advertising. After all the thing to be advertised will be binary digits only but the way these bits are to be interpreted by the scanner depends on the type of data the bits represent.

The type of data in BLE specs is governed by the UUID which stands for Universally Unique ID. UUIDs are of different types (learn more about them here). The type of UUID we are going to use is a 16 byte UUID.

Since the UUID is 16 bytes long, it can practically be any random number. So go ahead make up any 16 bytes of imaginary data, write them up in hex like so:

63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5

The command used for sending out data is

sudo hcitool -i hci0 cmd 0x08 0x0008

But it takes a few arguments, one of which is the UUID that we have just covered, now lets look at the rest of the arguments.

The first byte in the argument list is the length of the entire packet to be sent. Lets just assume it to be XX for now, once we have figured out the entire packet then we can come back and fill in the length accordingly.

The rest of the packet is made of structures called AD structures. An advertising packet basically contains, the length field (XX for now) followed by any arbitrary number of AD structures each having a special purpose. An AD structure has the following format:

1 byte for the length of the structure following the length byte (assume this to be n)

1 byte for the type of the AD structure. There are certain types of these structures, each type representing a purpose.

n -1 bytes of data.

Our packet is going to have 2 AD structures.

The first AD structure is to set certain flags. Such AD structures that are used configure flags have the type 01. The value of the structure is the flag mask to be used to configure the flags. The flag mask is only 1 byte long and I am not even going to pretend that I know anything about what these flags mean. You're just going to have to trust me with the value and the value is 1A. So in total the length should be 02 as that is the number of bytes in this AD structure after the first length byte.

The next and last AD structure will eventually contain the payload of this packet which constitutes the value and UUID. We will calculate the length after all the other bytes have been accounted for in this structure. The type of this AD structure is FF. It represents Manufacturer information. The value of this AD structure contains the company ID followed by the payload. The company ID is a two byte code assigned by Bluetooth SIG to top semiconductor industries in the world. The code for STMicroelectronics for example is 00 30 (to be written in the reverse order in the packet). After the company code, there is a byte that represents the number of bytes remaining from here in out in this AD structure. The value for this is 15 in hex which equals to 21 more bytes. 16 of these 21 bytes are the UUID. Following the UUID is your 4 byte data which can be anything of your choice (I am going to use 00 00 00 00). The last byte in the structure is the Tx power with which this signal is to be transmitted.

Let me put this in order and then we can count the number of bytes and substitute that for XX.

XX 02 01 1A 1A FF 30 00 02 15 63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5 00 00 00 00 C8

So these are 30 bytes following one byte for length. Hence the first byte should be the hex representation for 30 i.e. 1E. Now, finally our command looks like this:

sudo hcitool -i hci0 cmd 0x08 0x0008 1E 02 01 1A 1A FF 30 00 02 15 63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5 00 00 00 00 C8

You can now scan the beacon from other BLE devices. I would recommend using the beacon scanner app for Android.

To stop advertising use the following command:

sudo hciconfig hci0 noleadv

Checking availability of BLE in Linux

Most modern PCs have Bluetooth as part of their hardware. But not all of them are Bluetooth Low Energy enabled. I am going to describe a quick way to determine whether your Linux machine is BLE enabled or not. 

First thing to check is to see if the machine is Bluetooth enabled. To do so, follow this link.

After confirming that Bluetooth is there and is running, you are ready to enter into the world of BLE provided of course you're able to.

Open up a terminal and type the following command, assuming hci0 is your adaptor:

$ hciconfig hci0 lestates

This command will return one of two kind of output depending on whether BLE is present or not. If you get something like this:

Read LE supported states on hci0 returned status 1

Then this means that BLE is not present and this is not a limitation you can fix by tweaking software. You need to purchase a Bluetooth Low Energy adaptor and plug it in to your machine. 

On the spin side, if you see a list of states offered by your machine then that obviously mean BLE is good to go. The output may look something like:

Supported link layer states:

YES Non-connectable Advertising State

YES Scannable Advertising State

YES Connectable Advertising State

YES Directed Advertising State

YES Passive Scanning State

YES Active Scanning State

YES Initiating State/Connection State in Master Role

YES Connection State in the Slave Role

YES Non-connectable Advertising State and Passive Scanning State combination

YES Scannable Advertising State and Passive Scanning State combination

YES Connectable Advertising State and Passive Scanning State combination

YES Directed Advertising State and Passive Scanning State combination

YES Non-connectable Advertising State and Active Scanning State combination

YES Scannable Advertising State and Active Scanning State combination

YES Connectable Advertising State and Active Scanning State combination

YES Directed Advertising State and Active Scanning State combination

YES Non-connectable Advertising State and Initiating State combination

YES Scannable Advertising State and Initiating State combination

YES Non-connectable Advertising State and Master Role combination

YES Scannable Advertising State and Master Role combination

YES Non-connectable Advertising State and Slave Role combination

YES Scannable Advertising State and Slave Role combination

YES Passive Scanning State and Initiating State combination

YES Active Scanning State and Initiating State combination

YES Passive Scanning State and Master Role combination

YES Active Scanning State and Master Role combination

YES Passive Scanning State and Slave Role combination

YES Active Scanning State and Slave Role combination

YES Initiating State and Master Role combination/Master Role and Master Role combination

Receiving files from a phone using Bluetooth on a Linux machine

Following the other posts in this genre, I am going to describe a way to receive files sent from a mobile phone to a Linux machine using Bluetooth and as advertised, everything is going to be directly from the terminal.

First of all you need to look for a Bluetooth service running on the Linux machine. To do so, list the services out using the following command:

$ sudo sdptool browse local | grep "Service Name"

This will list out all the Bluetooth services offered presently by the machine. In order to receive files we need the service: "Object Push Profile". If it is listed then all is set otherwise you need to install it by either following this tutorial or simply following these commands:

$ sudo apt-get install obexpushd

Check if the obex-data process is still running:

$ ps auwx | grep obex-data

This will return a process id if the process is running otherwise nothing will be returned. Assuming the process id is XXXX, apply:

$ kill -9 XXXX

Start the service now using the following command:

$ sudo obexpushd -B -n

It should give the following output:

This software comes with ABSOLUTELY NO WARRANTY.

        This is free software, and you are welcome to redistribute it

        under certain conditions.

        Listening on bluetooth/[00:00:00:00:00:00]:9

        

Keep this thing running and open another terminal. Following the procedure given in this post, connect with the phone you wish to communicate with.

Now it is time to send files from your phone. Your machine will be ready to receive the files by now. When the transfer is done, you can quit the interactive session and kill the obexpushd process by doing ctrl + c.

Sending files using Bluetooth from terminal to an Android phone

In the last post I described how to get Bluetooth up and running on a Linux machine all from the terminal. In this post I am going to describe one half of the most obvious use case of Bluetooth i.e. sending files from the Linux machine to another Bluetooth device e.g. a phone.

I am assuming that you have a Bluetooth adaptor running, if no then check the above mentioned post on how to do that. 

First thing you are going to do has little to do with your Linux machine. You need to make your phone a ftp server. Don't worry, it's not very fancy. An app installation will do just fine. The app I'd recommend for this is the Astro Bluetooth Module by Metago. Once the app is installed, open it and tick the following three things (you are gonna wanna do this every time you need to receive files):

• Bluetooth
• Discover-able
• OBEX ftp server

OBEX is the Object Exchange profile that is designed specially for the transfer of files between two Bluetooth devices.

Now it is time to turn to the Linux machine and make sure it is ftp ready. First thing is to just update stuff just for good measure:

$ sudo apt-get update

Then you can go ahead and add obexftp:

$ sudo apt-get install obexftp

Now it is time to hunt for the phone you want to interact with. Use the following command to scan all nearby Bluetooth devices:

$ hcitool scan

It will start scanning like so:

Scanning ...

B4:CE:F6:B5:45:D7 Nexus 9

D8:5D:E2:A8:0B:B4 DIPTENDU-PC

5C:E0:C5:56:D3:CF PRIYARAN-E7450

You need to copy or at least make a note of the address of the device you wish to communicate with. Lets assume the address of the device is B4:CE:F6:B5:45:D7.

After this you're going to use an interactive tool provided to you already in order to do any interactions using Bluetooth. The tool is bluetoothctl and you can invoke it using the command:

$ bluetoothctl

Trust me and type the following four commands directly into the interactive session:

[bluetooth]# power on

[bluetooth]# agent on

[bluetooth]# default-agent

[bluetooth]# scan on

Now connect to the device using its MAC address:

[bluetooth]# pair B4:CE:F6:B5:45:D7

[bluetooth]# trust B4:CE:F6:B5:45:D7

Obviously you'll get some kind of verification code on your phone to confirm this connection, accept the invitation. You have now paired with the phone. Without closing this terminal, open up another one. And type the following command:

$ obexftp -b B4:CE:F6:B5:45:D7 -p /path/to/file/to/be/send

From here you should get a prompt at your phone showing the transfer happening.

Once the file transfer is done, exit the terminal window and quite the interactive session by typing:

[bluetooth]# quit

To know more about these interactive commands, you can use 

[bluetooth]# help

Finding, enabling and starting Bluetooth in Linux from terminal

First thing that is needed to be done is to check if your machine has a Bluetooth adaptor or not. Most modern PC should have one builtin but if that is not the case one can always buy a cheap Bluetooth USB dongle and plug it in. Once the hardware side of things are done with, enter the following command to start the essential software routines to handle Bluetooth:

$ sudo service bluetooth start

To know what and how many adaptors you have at your disposal you need to use the following command:

$ hciconfig

The above command will return nothing if no Bluetooth adaptor is available, else you will see something like this as output:

hci0: Type: BR/EDR  Bus: UART

BD Address: B8:27:EB:E4:87:C3  ACL MTU: 1021:8  SCO MTU: 64:1

UP RUNNING

RX bytes:773 acl:0 sco:0 events:50 errors:0

TX bytes:2540 acl:0 sco:0 commands:50 errors:0

What you see above is the primary information about the adaptor hci0. The zero here is like a serial number, if the system had two adaptors then there would have been two such outputs following hci0 and hci1. For the rest of the commands in this post I am going to use hci0 as a default.

If you wish to know more about the adaptor then use:

$ hciconfig -a hci0

In the output of the above commands you see in the second line something written as "UP RUNNING". This represents the current state of the adaptor. Currently hci0 is running. To shut the radio down use:

$ hciconfig hci0 down

To bring it back again:

$ hciconfig hci0 up

That is it. Bluetooth is now functional. As a validation you can scan using other devices to see if your device gets listed there. It should.

Once you're able to get this and are willing to continue using Bluetooth from the terminal, you need to install the following things for a better experience:

$ sudo apt-get install bluez

$ sudo apt-get install bluez-tools

BKL: One lock to bind them all

Synchronised access by multiple resources is an ability any decent OS kernel must posses. The access is not intended for just the user. In fact the kernel internals are the largest consumers of synchronised access.

The kernel has to micro manage the whole hardware of the machine as such you'd expect it to be intensely intertwined and cutting corners in terms of performance at all stages. Even if performance and efficiency are very much desired, one thing that cannot be compromised is the security and safety of it all. A common backdoor in the safe working of the kernel is often a vulnerability of the concurrency of its functions. So before one considers being clever in infinitely many ways for the purposes of improving the kernel's performance, one must be singularly confidant that the kernel is safely concurrent. A kernel not using the hardware to its full potential is ordinary but a kernel risking reliability by not preventing the hardware from over stepping processes is criminal. It goes without saying that locks are absolutely essential in a kernel.

I know almost nothing about kernels. Thanks to the Linux kernel, that almost wasn't an absolutely. Not long before the present versions of Linux, the famous kernel had an old companion called the big kernel lock. It belongs to the spin lock category. The reason for the use of the article "the" in its name is because of its exclusiveness. It was about the only lock the Linux kernel employed or needed to hold its own at concurrency of operations. The adjective "big" states the obvious that to manage a big piece of software, a big lock is needed. The bigness of a lock in my guess is estimated by the size of the critical section it holds, hence the name the big kernel lock is justified. 

The big kernel lock was a global spin lock responsible for most synchronisation efforts in the Linux kernel back when it was active and acting. Remember what a lock is? If no then this is awkward otherwise following are some interesting features of the big kernel lock:

1. A process can sleep and probably dream while holding the BKL. This means that a process doesn't have to relinquish the lock (which it probably should), but the lock will be released automatically anticipating the process' inactivity.
2. The lock is recursive in nature meaning that a single process can have consecutive and successful attempts at locking the kernel (or at least a big significant chunk of it).
3. Like the ring in the Lord of the rings, you don't want this as it corrupts, eventually.

There has been radical changes in the Linux kernel to remove the lock and although it hasn't completely been removed but its usage has been reduced dramatically after 2011. New code is discouraged from using the now infamous lock_kernel() API.

The BKL does not allow kernel preemption. This is a good thing when it is a good thing. I mean BKL is essential for cases where it remains used in the kernel still but it cannot be used extensively for locking the kernel. Doing so would be like jamming the whole city's traffic even if a small gathering was to happen in a corner of the town. A more fine graded locking mechanism is apt for such purposes. Such fine grading has been gradually applied in the kernel since. BKL is obsolete but it isn't extinct. For more informed and authentic information, read this link.

Locks and Types

Locking is essential for computers for the same reasons door knobs are essential for bathrooms. A lock is meant to ensure only one (or any other number) of contenders access a resource. By access I meant exploit and by contenders I meant entities that need the resource ideally immediately.

I mentioned how the number of entities accessing a lock can be more than one. This is because a resource may have more than one available copy of it self for the greater good of the consumer kind. The multiplicity of same resources is there for the same reasons that explain multiple compartments in a public toilet. In short it is good to build more hoping that no one has to wait much and hence no one has to explode. I was talking about computers if the above metaphors got mixed a little to homogeneously.

The basic idea is that a lock protects access to some kind of shared resource. If someone has the lock, that someone can use the resource. If someone does not have the lock, they have to wait. In order to "have the lock", one needs to wait for their turn in perhaps a queue or a skewed web of priorities (just like life is). Once the turn comes, the following abstract sequence follows:

1. Acquire the lock on the resource.
2. Do your thing, quickly but to your satisfaction.
3. Hope no one important comes and steals your turn in between "the act".
4. Release the lock when done, or release it anyway and wait till your turn comes back again.

The concept of locks is not just to let someone have protected access to a resource but is also to keep others from rushing into a busy process. Such waiting folks are said to have been blocked, temporarily.

Now moving on from what are locks to how they are implemented. Implementing a lock is as simple as counting stuff. You just need to count how many resources are available and how many are free. A lockable resource will have a maximum number of owners or lock holders. If that number has been reached then no one will be allowed to use it further simply because no one can. If and when a resource frees up, the next in line is allowed to access it. Simple. 

Enough for the overly simplified analogies for locks. Let us look at some types of locks:

1. Mutexes: Mutex is short for Mutual Exclusion. The word exclusion is anti to the whole idea of sharing. So mutex are locks that allow exclusive access to a resource to only one process at a time. Unless the degree of mutexness is diluted by excuses like shared mutex, recursive mutex or read/write mutex; a mutex most certainly means no sharing at all.
2. Recursice Mutex: It is the same as a regular mutex but more than one lock can be held on the resource at the same time. So they are nothing like mutex at all. No, but seriously they are, maybe not by intent but by principle. These allow the same process to hold multiple locks over the same resource, recursively. The complete freedom of the resource would mean an completely inverse unwinding of these locks. 
3. Reader/Writer Mutexes: These mutexes add more meaning to the notion of holding a lock over a resource. They classify locks to be of two kinds:
   1. Read locks: Locks given to threads that only wish to read the resource.
   2. Write locks: Locks given to threads that wish to make changes to the resource.
      If some process has a write lock at a resource then no one else is allowed to either change the resource or even read it. But if some process is having a read lock over the resource then other processes can also have a sneak peak and acquire read locks of their own.
4. Spinlocks: A spinlock is a special kind of mutex. Why? Probably since it does not need to be appended by the word mutex to have some respectable identity. In fact there was a spinlock in the Linux kernel that was so fundamental and so fundamentally wrong that they actually had to remove it or Linux would not have been so sophisticated.
   A spinlock does not use OS synchronisation functions when a contending process has to wait for its turn. Instead, it just keeps trying to update the mutex data structure to take the lock in a loop. It is the relentless type. This type of technique resonates with the inpatient types and hence works well when the locks are generally held for a short time and are changed hands frequently. However if some lock is in place and is meant to stay in place for a long time then such consistent looping to get the lock may seem and is a waste of time for the processor.
5. Semaphores: Semaphores are like mutex made rich and diplomatic. They have the powers of a mutex minus the rudeness. They deliberately interact with multiple processes and multiple resources. With each resource there is a number representing how many processes are currently consuming the resource and how many are currently craving it. I have just the post for you if you wish to read more about them. Read more about them.