Init hw9

files/hw9.txt

@@ -0,0 +1,14 @@
+./hw9/README.md
+./hw9/simu1/QUESTIONS.md
+./hw9/simu1/README.md
+./hw9/simu1/main-common.c
+./hw9/simu1/main-header.h
+./hw9/simu1/mythreads.h
+./hw9/simu1/vector-avoid-hold-and-wait.c
+./hw9/simu1/vector-deadlock.c
+./hw9/simu1/vector-global-order.c
+./hw9/simu1/vector-header.h
+./hw9/simu1/vector-nolock.c
+./hw9/simu1/vector-try-wait.c
+./hw9/task1/README.md
+./hw9/task2/README.md

hw9/README.md

@@ -0,0 +1,33 @@
+# hw9
+
+## Tasks
+
+To fulfill **hw9** you have to solve:
+
+- task1
+- task2
+- simu1
+
+## Files
+
+You find already or reuse some files for rust tasks. Please remember to
+use cargo to create a new project for each task and copy the given files
+into their dedicated directories.
+
+
+## Pull-Request
+
+Please merge any accepted reviews into your branch. If you are ready
+with the homework, all tests run, please create a pull request named
+
+**hw9**.
+
+## hw9 credits
+
+| Task     | max. Credits | Comment                 |
+|:---------|:-------------|:------------------------|
+| task1    | 1            |  |
+| simu1    | 1,5          |                         |
+| task2    | 1,5          |                         |
+| Deadline | +1           |                         |
+| =        | 5            |                         |

hw9/simu1/Makefile

@@ -0,0 +1,24 @@
+
+ALL = vector-deadlock vector-global-order vector-try-wait vector-avoid-hold-and-wait vector-nolock
+COMMON = vector-header.h main-common.c main-header.h
+
+all: $(ALL)
+
+clean:
+	rm -f $(ALL) *~
+
+vector-deadlock: vector-deadlock.c $(COMMON)
+	gcc -o vector-deadlock vector-deadlock.c -Wall -pthread -O
+
+vector-global-order: vector-global-order.c $(COMMON)
+	gcc -o vector-global-order vector-global-order.c -Wall -pthread -O
+
+vector-try-wait: vector-try-wait.c $(COMMON)
+	gcc -o vector-try-wait vector-try-wait.c -Wall -pthread -O
+
+vector-avoid-hold-and-wait: vector-avoid-hold-and-wait.c $(COMMON)
+	gcc -o vector-avoid-hold-and-wait vector-avoid-hold-and-wait.c -Wall -pthread -O
+
+vector-nolock: vector-nolock.c $(COMMON)
+	gcc -o vector-nolock vector-nolock.c -Wall -pthread -O
+

hw9/simu1/QUESTIONS.md

@@ -0,0 +1,74 @@
+# QUESTIONS Synchronisation: Real Bugs
+
+This homework lets you explore some real code that deadlocks (or avoids
+deadlock). The different versions of code correspond to different
+approaches to avoiding deadlock in a simplified vector add() routine.
+Specifically:
+
+- `vector-deadlock.c`: This version of vector add() does not try to
+  avoid deadlock and thus may indeed do so.
+- `vector-global-order.c`: This version acquires locks in a global order
+  to avoid deadlock.
+- `vector-try-wait.c`: This version is willing to release a lock when it
+  senses deadlock might occur.
+- `vector-avoid-hold-and-wait.c`: This version uses a global lock around
+  lock acquisition to avoid deadlock.
+- `vector-nolock.c`: This version uses an atomic fetch-and-add instead
+  of locks.
+
+See the README for details on these programs and their common substrate.
+
+## Questions
+
+1. First let’s make sure you understand how the programs generally work,
+   and some of the key options. Study the code in the file called
+   `vector-deadlock.c`, as well as in `main-common.c` and related files.
+
+   Now, run **./vector-deadlock -n 2 -l 1 -v**, which instantiates two
+   threads (`-n 2`), each of which does one vector add (`-l 1`), and
+   does so in verbose mode (`-v`). Make sure you understand the output.
+   How does the output change from run to run?
+
+2. Now add the `-d` flag, and change the number of loops (`-l`) from 1
+   to higher numbers. What happens? Does the code (always) deadlock?
+
+3. How does changing the number of threads (`-n`) change the outcome of
+   the program? Are there any values of `-n` that ensure no deadlock
+   occurs?
+
+4. Now examine the code in `vector-global-order.c`. First, make sure you
+   understand what the code is trying to do; do you understand why the
+   code avoids deadlock? Also, why is there a special case in this
+   vector *add()* routine when the source and destination vectors are
+   the same?
+
+5. Now run the code with the following flags: `-t -n 2 -l 100000 -d`.
+   How long does the code take to complete? How does the total time
+   change when you increase the number of loops, or the number of
+   threads?
+
+6. What happens if you turn on the parallelism flag (`-p`)? How much
+   would you expect performance to change when each thread is working on
+   adding different vectors (which is what `-p` enables) versus working
+   on the same ones?
+
+7. Now let’s study `vector-try-wait.c`. First make sure you understand
+   the code. Is the first call to pthread mutex `trylock()` really
+   needed?
+
+   Now run the code. How fast does it run compared to the global order
+   approach? How does the number of retries, as counted by the code,
+   change as the number of threads increases?
+
+8. Now let’s look at `vector-avoid-hold-and-wait.c`. What is the main
+   problem with this approach? How does its performance compare to the
+   other versions, when running both with `-p` and without it?
+
+9. Finally, let’s look at `vector-nolock.c`. This version doesn’t use
+   locks at all; does it provide the exact same semantics as the other
+   versions? Why or why not?
+
+10. Now compare its performance to the other versions, both when threads
+    are working on the same two vectors (no `-p`) and when each thread
+    is working on separate vectors (`-p`). How does this no-lock version
+    perform?

hw9/simu1/README.md

@@ -0,0 +1,82 @@
+# README Synchronisation: Real Bugs
+
+This homework lets you play around with a number of ways to implement a
+small, deadlock-free vector object in C. The vector object is quite
+limited (e.g., it only has *add()* and *init()* functions) but is just
+used to illustrate different approaches to avoiding deadlock.
+
+Some files that you should pay attention to are as follows. They, in
+particular, are used by all the variants in this homework.
+
+- `mythreads.h` The usual wrappers around many different pthread (and
+  other) library calls, so as to ensure they are not failing silently
+
+- `vector-header.h` A simple header for the vector routines, mostly
+  defining a fixed vector size and then a struct that is used per vector
+  (vector_t)
+
+- `main-header.h` A number of global variables common to each different
+  program
+
+- `main-common.c` Contains the *main()* routine (with arg parsing) that
+  initializes two vectors, starts some threads to access them (via a
+  *worker()* routine), and then waits for the many *vector_add()*'s to
+  complete
+
+The variants of this homework are found in the following files. Each
+takes a different approach to dealing with concurrency inside a "vector
+addition" routine called *vector_add()*; examine the code in these files
+to get a sense of what is going on. They all use the files above to make
+a complete runnable program.
+
+- `vector-deadlock.c` This version blithely grabs the locks in a
+  particular order (dst then src). By doing so, it creates an
+  "invitation to deadlock", as one thread might call *vector_add(v1,
+  v2)* while another concurrently calls *vector_add(v2, v1)*.
+
+- `vector-global-order.c` This version of *vector_add()* grabs the locks
+  in a total order, based on address of the vector.
+
+- `vector-try-wait.c` This version of *vector_add()* uses
+  *pthread_mutex_trylock()* to attempt to grab locks; when the try
+  fails, the code releases any locks it may hold and goes back to the
+  top and tries it all over again.
+
+- `vector-avoid-hold-and-wait.c` This version of *vector_add()* ensures
+  it can't get stuck in a hold and wait pattern by using a single lock
+  around lock acquisition.
+
+- `vector-nolock.c` This version of *vector_add()* doesn't even use
+  locks; rather, it uses an atomic fetch-and-add to implement the
+  *vector_add()* routine. Its semantics (as a result) are slightly
+  different.
+
+Type "make" (and read the Makefile) to build each of five executables.
+
+    prompt> make
+
+Then you can run a program by simply typing its name:
+
+    prompt> ./vector-deadlock
+
+Each program takes the same set of arguments (see `main-common.c` for
+details):
+
+`-d` This flag turns on the ability for threads to deadlock. When you
+pass -d to the program, every other thread calls *vector_add()* with the
+vectors in a different order, e.g., with two threads, and -d enabled,
+Thread 0 calls *vector_add(v1, v2)* and Thread 1 calls *vector_add(v2,
+v1)*
+
+`-p` This flag gives each thread a different set of vectors to call
+*add()* upon, instead of just two vectors. Use this to see how things
+perform when there isn't contention for the same set of vectors.
+
+`-n num_threads` Creates some number of threads; you need more than one
+to deadlock.
+
+`-l loops` How many times should each thread call *vector_add()*?
+
+`-v` Verbose flag: prints out a little more about what is going on.
+
+`-t` Turns on timing and shows how long everything took.

hw9/simu1/main-common.c

@@ -0,0 +1,139 @@
+
+vector_t v[2 * MAX_THREADS];
+
+// used to ensure print outs make sense
+pthread_mutex_t print_lock = PTHREAD_MUTEX_INITIALIZER;
+
+void usage(char *prog) {
+    fprintf(stderr, "usage: %s [-d (turn on deadlocking behavior)] [-l loops] [-n num_threads] [-t (do timing)] [-v (for verbose)]\n", prog);
+    exit(1);
+}
+
+// only called by one thread (not thread-safe)
+void vector_init(vector_t *v, int value) {
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	v->values[i] = value;
+    }
+    Pthread_mutex_init(&v->lock, NULL); 
+}
+
+// only called by one thread (not thread-safe)
+void vector_print(vector_t *v, char *str) {
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	printf("%s[%d] %d\n", str, i, v->values[i]);
+    }
+}
+
+void print_info(int call_return, int thread_id, int v0, int v1) {
+    if (verbose == 0)
+	return;
+    Pthread_mutex_lock(&print_lock);
+    int j;
+    for (j = 0; j < thread_id; j++) printf("              ");
+    if (call_return) 
+	printf("<-add(%d, %d)\n", v0, v1);
+    else
+	printf("->add(%d, %d)\n", v0, v1);
+    Pthread_mutex_unlock(&print_lock);
+}
+
+typedef struct __thread_arg_t {
+    int tid;
+    int vector_add_order;
+    int vector_0;
+    int vector_1;
+} thread_arg_t;
+
+void *worker(void *arg) {
+    thread_arg_t *args = (thread_arg_t *) arg;
+    int i, v0, v1;
+    for (i = 0; i < loops; i++) {
+	if (args->vector_add_order == 0) { 
+	    v0 = args->vector_0;
+	    v1 = args->vector_1;
+	} else {
+	    v0 = args->vector_1;
+	    v1 = args->vector_0;
+	}
+
+	print_info(0, args->tid, v0, v1);
+
+	vector_add(&v[v0], &v[v1]);
+
+	print_info(1, args->tid, v0, v1);
+    }
+    return NULL;
+}
+
+int main(int argc, char *argv[]) {
+    opterr = 0;
+    int c;
+    while ((c = getopt (argc, argv, "l:n:vtdp")) != -1) {
+	switch (c) {
+	case 'l':
+	    loops = atoi(optarg);
+	    break;
+	case 'n':
+	    num_threads = atoi(optarg);
+	    break;
+	case 'v':
+	    verbose = 1;
+	    break;
+	case 't':
+	    do_timing = 1;
+	    break;
+	case 'd':
+	    cause_deadlock = 1;
+	    break;
+	case 'p':
+	    enable_parallelism = 1;
+	    break;
+	default:
+	    usage(argv[0]);
+	}
+    }
+
+    assert(num_threads < MAX_THREADS);
+
+    pthread_t pid[MAX_THREADS];
+    thread_arg_t args[MAX_THREADS];
+    int i;
+    for (i = 0; i < num_threads; i++) {
+	args[i].tid = i;
+	if (enable_parallelism == 0) {
+	    args[i].vector_0 = 0;
+	    args[i].vector_1 = 1;
+	} else {
+	    args[i].vector_0 = i * 2;
+	    args[i].vector_1 = i * 2 + 1;
+	}
+
+	if (cause_deadlock && i % 2 == 1)
+	    args[i].vector_add_order = 1;
+	else
+	    args[i].vector_add_order = 0;
+    }
+
+    for (i = 0; i < 2 * MAX_THREADS; i++) 
+	vector_init(&v[i], i);
+
+    double t1 = Time_GetSeconds();
+
+    for (i = 0; i < num_threads; i++) 
+	Pthread_create(&pid[i], NULL, worker, (void *) &args[i]);
+    for (i = 0; i < num_threads; i++) 
+	Pthread_join(pid[i], NULL); 
+
+    double t2 = Time_GetSeconds();
+
+    fini();
+    if (do_timing) {
+	printf("Time: %.2f seconds\n", t2 - t1);
+    }
+
+    //vector_print(&v[0], "v1");
+    //vector_print(&v[1], "v2");
+    return 0;
+}

hw9/simu1/main-header.h

@@ -0,0 +1,13 @@
+#ifndef __main_header_h__
+#define __main_header_h__
+
+#define MAX_THREADS (100)
+
+int loops = 1;
+int verbose = 0;
+int num_threads = 2;
+int do_timing = 0;
+int cause_deadlock = 0;
+int enable_parallelism = 0;
+
+#endif // __main_header_h__

hw9/simu1/mythreads.h

@@ -0,0 +1,52 @@
+#ifndef __MYTHREADS_h__
+#define __MYTHREADS_h__
+
+#include <pthread.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <sys/time.h>
+
+void *Malloc(size_t size) {
+    void *rc = malloc(size);
+    assert(rc != NULL);
+    return rc;
+}
+
+double Time_GetSeconds() {
+    struct timeval t;
+    int rc = gettimeofday(&t, NULL);
+    assert(rc == 0);
+    return (double) ((double)t.tv_sec + (double)t.tv_usec / 1e6);
+}
+
+void Pthread_mutex_init(pthread_mutex_t *mutex,
+			const pthread_mutexattr_t *attr) {
+    int rc = pthread_mutex_init(mutex, attr);
+    assert(rc == 0);
+}
+
+
+void Pthread_mutex_lock(pthread_mutex_t *m) {
+    int rc = pthread_mutex_lock(m);
+    assert(rc == 0);
+}
+                                                                                
+void Pthread_mutex_unlock(pthread_mutex_t *m) {
+    int rc = pthread_mutex_unlock(m);
+    assert(rc == 0);
+}
+                                                                                
+void Pthread_create(pthread_t *thread, const pthread_attr_t *attr, 	 
+		    void *(*start_routine)(void*), void *arg) {
+    int rc = pthread_create(thread, attr, start_routine, arg);
+    assert(rc == 0);
+}
+
+void Pthread_join(pthread_t thread, void **value_ptr) {
+    int rc = pthread_join(thread, value_ptr);
+    assert(rc == 0);
+}
+
+
+
+#endif // __MYTHREADS_h__

hw9/simu1/vector-avoid-hold-and-wait.c

@@ -0,0 +1,32 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <unistd.h>
+
+#include "mythreads.h"
+
+#include "main-header.h"
+#include "vector-header.h"
+
+// use this to make lock acquisition ATOMIC
+pthread_mutex_t global = PTHREAD_MUTEX_INITIALIZER; 
+
+void vector_add(vector_t *v_dst, vector_t *v_src) {
+    // put GLOBAL lock around all lock acquisition...
+    Pthread_mutex_lock(&global);
+    Pthread_mutex_lock(&v_dst->lock);
+    Pthread_mutex_lock(&v_src->lock);
+    Pthread_mutex_unlock(&global);
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	v_dst->values[i] = v_dst->values[i] + v_src->values[i];
+    }
+    Pthread_mutex_unlock(&v_dst->lock);
+    Pthread_mutex_unlock(&v_src->lock);
+}
+
+void fini() {}
+
+#include "main-common.c"
+

hw9/simu1/vector-deadlock.c

@@ -0,0 +1,24 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "mythreads.h"
+
+#include "main-header.h"
+#include "vector-header.h"
+
+void vector_add(vector_t *v_dst, vector_t *v_src) {
+    Pthread_mutex_lock(&v_dst->lock);
+    Pthread_mutex_lock(&v_src->lock);
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	v_dst->values[i] = v_dst->values[i] + v_src->values[i];
+    }
+    Pthread_mutex_unlock(&v_dst->lock);
+    Pthread_mutex_unlock(&v_src->lock);
+}
+
+void fini() {}
+
+#include "main-common.c"

hw9/simu1/vector-global-order.c

@@ -0,0 +1,35 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "mythreads.h"
+
+#include "main-header.h"
+#include "vector-header.h"
+
+void vector_add(vector_t *v_dst, vector_t *v_src) {
+    if (v_dst < v_src) {
+	Pthread_mutex_lock(&v_dst->lock);
+	Pthread_mutex_lock(&v_src->lock);
+    } else if (v_dst > v_src) {
+	Pthread_mutex_lock(&v_src->lock);
+	Pthread_mutex_lock(&v_dst->lock);
+    } else {
+	// special case: src and dst are the same
+	Pthread_mutex_lock(&v_src->lock);
+    }
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	v_dst->values[i] = v_dst->values[i] + v_src->values[i];
+    }
+    Pthread_mutex_unlock(&v_src->lock);
+    if (v_dst != v_src) 
+	Pthread_mutex_unlock(&v_dst->lock);
+}
+
+void fini() {}
+
+
+#include "main-common.c"
+

hw9/simu1/vector-header.h

@@ -0,0 +1,13 @@
+#ifndef __vector_header_h__
+#define __vector_header_h__
+
+#define VECTOR_SIZE (100)
+
+typedef struct __vector {
+    pthread_mutex_t lock;
+    int values[VECTOR_SIZE];
+} vector_t;
+
+
+#endif // __vector_header_h__
+

hw9/simu1/vector-nolock.c

@@ -0,0 +1,31 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "mythreads.h"
+
+#include "main-header.h"
+#include "vector-header.h"
+
+// taken from https://en.wikipedia.org/wiki/Fetch-and-add
+int fetch_and_add(int * variable, int value) {
+    asm volatile("lock; xaddl %%eax, %2;"
+		 :"=a" (value)                  
+		 :"a" (value), "m" (*variable)  
+		 :"memory");
+    return value;
+}
+
+void vector_add(vector_t *v_dst, vector_t *v_src) {
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	fetch_and_add(&v_dst->values[i], v_src->values[i]);
+    }
+}
+
+void fini() {}
+
+
+#include "main-common.c"
+

hw9/simu1/vector-try-wait.c

@@ -0,0 +1,36 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "mythreads.h"
+
+#include "main-header.h"
+#include "vector-header.h"
+
+int retry = 0;
+
+void vector_add(vector_t *v_dst, vector_t *v_src) {
+  top:
+    if (pthread_mutex_trylock(&v_dst->lock) != 0) {
+	goto top;
+    }
+    if (pthread_mutex_trylock(&v_src->lock) != 0) {
+	retry++;
+	Pthread_mutex_unlock(&v_dst->lock);
+	goto top;
+    }
+    int i;
+    for (i = 0; i < VECTOR_SIZE; i++) {
+	v_dst->values[i] = v_dst->values[i] + v_src->values[i];
+    }
+    Pthread_mutex_unlock(&v_dst->lock);
+    Pthread_mutex_unlock(&v_src->lock);
+}
+
+void fini() {
+    printf("Retries: %d\n", retry);
+}
+
+#include "main-common.c"
+

hw9/task1/README.md

@@ -0,0 +1,106 @@
+# Homework hw9 task1
+
+[TOC]: #
+
+# Table of Contents
+- [1.1. Ziel](#11-ziel)
+- [1.2. Protokollbeschreibung](#12-protokollbeschreibung)
+    - [1.2.1 STAGE](#121-stage)
+    - [1.2.2 RETRIEVE](#122-retrieve)
+- [2.1 Aufgabe](#21-aufgabe)
+    - [2.2.1 Crate](#221-crate)
+    - [2.2.2 Testabdeckung](#222-testabdeckung)
+    - [2.2.3 Fehlerbehandlung und Rückgabetyp](#223-fehlerbehandlung-und-rückgabetyp)
+- [3.1 Hilfen](#31-hilfen)
+
+
+## 1.1. Ziel
+
+Ziel dieser Aufgabe ist es, einen Parser für ein simples Protokoll zu
+implementieren, das in den folgenden Aufgaben verwendet wird.
+
+Das Protokoll beschreibt das Bereitstellen und Abrufen von Daten von
+einem Server.
+
+## 1.2. Protokollbeschreibung
+
+Das Protokoll kennt 2 Kommandos: Bereitstellen und Abrufen. Weitere
+Kommandos sind zunächst nicht zugelassen. Das Protokoll arbeitet
+zeilenweise, d.h. ein newline ('\\n') beendet das Kommando.
+
+Die Kommandos folgen diesem Format:
+
+    <KOMMANDO> [DATEN]\n
+
+Werden keine Daten erwartet, folgt das newline-Zeichen direkt auf das
+Kommando. Das Kommando muss immer in Großbuchstaben geschrieben werden.
+
+Der Zeilenumbruch ist _nicht_ optional.
+
+### 1.2.1 STAGE
+
+Das STAGE-Kommando überträgt eine Nachricht an den Server. Die Daten
+dürfen selbst keine newline enthalten. Es ist erlaubt, eine _leere_
+Nachricht zu veröffentlichen.
+
+Beispiele:
+
+    STAGE Hallo, hier ist eine kleine Nachricht!\n
+    STAGE \n
+
+### 1.2.2 RETRIEVE
+
+Das RETRIEVE-Kommando entnimmt eine Nachricht. Es erhält keine Daten.
+
+Beispiel:
+
+    RETRIEVE\n
+
+## 2.1 Aufgabe
+
+Implementieren Sie einen Parser für dieses Protokoll. Der Parser sollte so
+simpel wie möglich gehalten sein.
+
+Es ist insbesondere erlaubt, die Eingabe als einzelne Zeile zu erwarten.
+Die Trennung der Eingabe in einzelne Zeilen ist dem aufrufenden Code
+überlassen.
+
+Enthält der Input mehrere newlines, darf der Rest ohne Fehler verworfen
+weden.
+
+Folgendes öffentliche Interface ist vorgegeben:
+
+```rust
+pub fn parse(message: &str) -> ... {
+   //...
+}
+```
+
+### 2.2.1 Crate
+
+Implementieren Sie den Parser als eigene Crate, die Sie dann in den
+weiteren Aufgaben einbinden können, statt den Code zu kopieren. Sie
+können diese Crate später entweder auf Github veröffentlichen oder über
+Pfade einbinden.
+
+### 2.2.2 Testabdeckung
+
+Achten Sie auf eine ausreichende Testabdeckung, um möglichst alle
+illegalen Fälle abzudecken. Achten Sie vor allem darauf, dass keine
+Daten verloren gehen!
+
+### 2.2.3 Fehlerbehandlung und Rückgabetyp
+
+Finden Sie einen geeigneten Rückgabetyp, der das gefundene Kommando, als
+auch eventuelle Fehler kommuniziert. Kombinieren Sie dies mit einer
+geeigneten Darstellung des Kommandos.
+
+Allokieren Sie die gefunden Daten als `String`.
+
+## 3.1 Hilfen
+
+Die Dokumentation von [`str`][str] enthält viele wichtige Hinweise.
+
+
+[str]: https://doc.rust-lang.org/std/primitive.str.html
+