Completions

Many parts of the kernel initiate certain activities as separate execution threads and then wait for those threads to complete. The completions interface made just for that.

Look at the example:

static DECLARE_COMPLETION(my_thread_completion);
static DECLARE_WAIT_QUEUE_HEAD(my_wait_queue);
int request_to_end_thread = 0;

static int my_thread(void *unused)
{
 DECLARE_WAITQUEUE(wait, current);
 daemonize("my_thread");
 add_wait_queue(&my_wait_queue, &wait);

 while (1)
 {
  //Request to opt out of the scheduler run queue.
  set_current_state(TASK_INTERRUPTIBLE);

  //asks the scheduler to choose and run a new task from its run queue.
  schedule();

        /* The thread is put back into the scheduler run queue.
  This could only happend because of an event over the wait queue. */
  if (request_to_end_thread) 
  {
   break;
  }
 }

 //Bail out of the wait queue
 __set_current_state(TASK_RUNNING);
 remove_wait_queue(&my_wait_queue, &wait);

 //Atomically signal completion and exit
 complete_and_exit(&my_thread_completion, 0);
}

static int __init my_init(void)
{
 kernel_thread(my_thread, NULL, CLONE_FS | CLONE_FILES | CLONE_SIGHAND | SIGCHLD);
}

static void __exit my_release(void)
{
 request_to_end_thread = 1;
 wake_up(&my_wait_queue);

        //this function blocks until my_thread exit. 

        wait_for_completion(&my_thread_completion);
}

Note: There may be more then 1 waiter for the same completion, so the thread which notify its completion may use:
void complete_all(struct completion *c);
to notify all waiters.

Notifier Chains

A Linux kernel consists from many different subsystems.
Often one subsystem wants to be notified of something happening in another subsystem. Notifier Chains solve this problem by providing a way for different subsystems to subscribe on asynchronous events from other subsystems.

Examples of such notifications include the following:

1. Die notification - sent when a kernel function triggers a trap or a fault.
2. Net device notification - sent when a network interface goes up or down.
3. Internet address notification - Sent when IP is changed on a network interface.

A custom notifier may be written for some kind of a new event, example will be shown later...

To be able to receive notification, a user must register its handler function with a specific notifier.

A handler function signature is:
int some_handler(struct notifier_block *self, unsigned long val, void *data)

• val - type of event
• data - pointer to some data structure

Both value and data have different kind of content and it depends on the kind of event notifier.

For example, for - die event, val equals - 0 if it is an "oops" event, and data points to a structure which contains all CPU registers status.

Some example code:

#include <linux/notifier.h>
#include <asm/kdebug.h>
#include <linux/netdevice.h>
#include <linux/inetdevice.h>

/* Net Device notifier definition */
static struct notifier_block my_net_dev_notifier = 
{
 .notifier_call = my_net_dev_event_handler,
};

/* Net Device notification event handler */
int my_net_dev_event_handler(struct notifier_block *self, 

    unsigned long val, void *data)
{
 printk("my_net_dev_event: status=%ld, Interface=%s\n", val, 

        ((struct net_device *) data)->name);
 return 0;
}

static int __init my_init(void)
{
 // Register Net Device Notifier
 register_netdevice_notifier(&my_net_dev_notifier);
}

You can generate this event by changing a state of a network interface, for example:
$sudo ifconfig eth0 up

You should receive a message:
my_net_dev_event_handler: status=1, Interface=eth0

When status=1, it means NETDEV_UP event.

Some example of a custom of a custom notifier:

#include <linux/notifier.h>
#include <asm/kdebug.h>

/* User-defined notifier chain implementation */
static BLOCKING_NOTIFIER_HEAD(my_notif_chain);

static struct notifier_block my_notifier = 
{
 .notifier_call = my_event_handler,
};

/* User-defined notification event handler */
int my_event_handler(struct notifier_block *self, 

                        unsigned long val, void *data)
{
 printk("my_event_handler: status=%ld\n", val);
 return 0;
}

/* Driver Initialization */
static int __init my_init(void)
{
 // Register a user-defined Notifier
 blocking_notifier_chain_register(&my_notif_chain,  

                                    &my_notifier);
} 

You may invoke this new custom event anywhere in the code by calling:
blocking_notifier_call_chain(&my_notif_chain, 1000, NULL);
You should receive a message:
my_event_handler: status=1000

Note:
We have declared our notification chain using - BLOCKING_NOTIFIER_HEAD

and registered our notifier using blocking_notifier_chain_register() function. By doing that, we declared that the notifier will be called from a process and because of that the handler function is allowed to sleep.

Now, if your notifier can be called from interrupt context, declare the notifier chain using ATOMIC_NOTIFIER_HEAD(), and register it via atomic_notifier_chain_register().

Thats all for now... Thanks for reading!

Work queues

Hi,

Today I want to talk about work queues.
Work queues are used in situations when the caller cannot do the intended action itself, for instance because it is an interrupt service routine and the work is too long for an interrupt, or is otherwise inappropriate to run in an interrupt (because it requires a process context).

Your code that should run in a later time is called a "work".
"Work" is some action that should complete in a reasonable time, because multiple work items share the the same worker thread.

When you want to use the work queue mechanism, you have 3 options to create a kernel thread:

1. singlethread_workqueue() - creates a work queue on a single CPU.
2. create_workqueue() - To create one worker thread per CPU in the system.
3. Use the default work queue. These are per-CPU worker threads, which were created at the time of boot up, that you can timeshare.

A work can be submitted to a dedicated queue using queue_work(), or to the default kernel worker thread using schedule_work().

Example code:

#include <linux/workqueue.h>
 

struct workqueue_struct *wq;
 

static int __init mymodule_init(void)
{
    /* ... */
    wq = create_singlethread_workqueue("my_wq");
    return 0;
}

int add_work_to_my_wq(void (*func)(void *data), void *data)
{
    struct work_struct *some_work;
    some_work = kmalloc(sizeof(struct work_struct), GFP_KERNEL);
    /* Init the work structure */
    INIT_WORK(some_work, func, data);
    /* Enqueue Work */
    queue_work(wq, some_work);
    return 0;
}

User may submit work to a work queue with a delay request, using:

int queue_delayed_work(struct workqueue_struct *queue,

    struct work_struct *work, unsigned long delay);

NOTE: Work queues API is only available to modules which are declared under GPL license, using:

MODULE_LICENSE("GPL");

Kernel threads

So, today we will talk about Kernel Threads.

What are Kernel threads

The purpose of a kernel threads is to make a tasks which will run in the background. Usually kernel threads wait for some asynchronous events and then wake up to serve them. For example, the kswapd is kernel thread which runs in the background and wait for expiration of a swap timer, then it handles swapping memory to disk if the number of free pages in the system is low.
If you type the following in your terminal:
$ ps -aux

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0 185512  6100 ?        Ss   אוק06   0:04 /lib/systemd/systemd --system --deserialize 28
root         2  0.0  0.0      0     0 ?        S    אוק06   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    אוק06   0:00 [ksoftirqd/0]
root         5  0.0  0.0      0     0 ?        S<   אוק06   0:00 [kworker/0:0H]
root         7  0.0  0.0      0     0 ?        S    אוק06   0:23 [rcu_sched]
root         8  0.0  0.0      0     0 ?        S    אוק06   0:00 [rcu_bh]
root         9  0.0  0.0      0     0 ?        S    אוק06   0:00 [migration/0]
root        10  0.0  0.0      0     0 ?        S    אוק06   0:00 [watchdog/0]
root        11  0.0  0.0      0     0 ?        S    אוק06   0:00 [watchdog/1]
root        12  0.0  0.0      0     0 ?        S    אוק06   0:00 [migration/1]
root        13  0.0  0.0      0     0 ?        S    אוק06   0:00 [ksoftirqd/1]
...

All this processes which name is surrounded by square brackets are kernel threads.

Lets see some code:

Low level kernel thread usage

I will begin, by showing you a low level thread API, then we look at higher level API which should be usually used.

This is how to create a linux thread:

ret = kernel_thread(mythread, NULL,
CLONE_FS | CLONE_FILES | CLONE_SIGHAND | SIGCHLD);

There parameters we pass here tells which resources should be shared between the parent thread and the child thread. For example - CLONE_FILES, share opened files, CLONE_SIGHAND, share signal handlers.
Each thread in Linux has a single parent and usually the new created threads are re-parented to kthreadd thread, to avoid the new thread to become a zombie if a parent process dies without waiting for its child to exit.

static DECLARE_WAIT_QUEUE_HEAD(myevent_waitqueue); rwlock_t myevent_lock; static int mythread(void *unused) { unsigned int event_id = 0; DECLARE_WAITQUEUE(wait, current);   /* performs initial housekeeping and changes the parent of the calling thread  to a kernel thread called kthreadd */ daemonize("mythread");   /* all signals are blocked by default, so we need to enable a particular signal. */ allow_signal(SIGKILL);   /* The thread sleeps on this wait queue until it's woken up */ add_wait_queue(&myevent_waitqueue, &wait);   for (;;)  { /* Request to opt out of the scheduler run queue */ set_current_state(TASK_INTERRUPTIBLE);  /* asks the scheduler to choose and run a new task from its run queue */ schedule();     /* At this point scheduler executes another task and it will not execute the current kthread anymore. */   /* Die if SIGKILL received, it is the only signal that may be received... */ if (signal_pending(current))  { break; }   /* The thread is put back into the scheduler run queue. This could only happend because of an event over the wait queue. */ read_lock(&myevent_lock);      /* Do something here... */   read_unlock(&myevent_lock); }    /* Changed to TASK_RUNNING, so there is no race condition even if the wake up occurs  between the time the task state is set to TASK_INTERRUPTIBLE and the time schedule()  is called.   */ set_current_state(TASK_RUNNING);   remove_wait_queue(&myevent_waitqueue, &wait);   return 0; }

Notes about kthread

Kernel threads may be preempted if the kernel was compiled with CONFIG_PREEMPT flag.A code may prevent preemption, even of the CONFIG_PREEMPT flag, by disabling local interrupts.

A kernel thread can be in any of the following process states:

• TASK_RUNNING: In the scheduler run queue, so should run in future.
• TASK_INTERRUPTIBLE: Waiting for an event and is not in the
  scheduler run queue.
• TASK_UNINTERRUPTIBLE: Receipt of a signal will not put the task back into the run queue.
• TASK_STOPPED: Stopped execution as a result of receiving a signal.
• TASK_TRACED: If traced by application such as strace.
• EXIT_ZOMBIE: His parent didn't wait for it to complete.
• EXIT_DEAD: Exited gracefully.

set_current_state()

Selects kthread's state.

To wake up a thread which waits on a wait queue, we use wake_up_interruptible(&myevent_waitqueue) function.

The higher level thread API

This API is actually one of several kernel helper interfaces which exist to make a kernel programmer's life a little easier...

#include <linux/kthread.h>

static int my_thread(void *unused)
{
    DECLARE_WAITQUEUE(wait, current);

    while (!kthread_should_stop()) 
    {
        /* ... */
    }

    __set_current_state(TASK_RUNNING);
    remove_wait_queue(&my_thread_wait, &wait);

    return 0;
}

void my_thread_init(void)
{
    my_task = kthread_create(my_thread, NULL, "%s", "my_thread");
}

void my_thread_wakeup(void)
{
    if (my_task)
    {
        wake_up_process(my_task);
    }
}

void my_thread_release(void)
{
    kthread_stop(my_task);
}

This is a more proper way to handle threads.

This is for today, thanks for reading!

An embedded project from 2013

Few years ago, I recorded a video of my self, demonstrating what you can do at your home with a TI Launchpad MSP430 (5$) and few other sensors...
I think it is time to share that video.

PS: Sorry that I talk Hebrew on the video and not English, my next videos will be in English...

 

The birth of my new blog

Hello everyone,

My name is Arkadi Viner and today I have decided to start writing a blog.
I'm a software engineer, so I really like technology and everything in between (and doing sports, but this is a subject for another blog...)

This will be a technology oriented blog which may help some of you to control Linux, write a better code or find out about some really cool gadgets.

I hope you will like reading my posts and I will happily respond to any of your questions!