01/03 Syncs and Locks

• So even though Go prefers channels and go routines for concurrent tasks.
• There are cases where locking memory is inevitable.
• So in go to access any functionality related to thread Locking we need to use the sync package which is a standard package in go as shown here.
• So there are numerous features enabled by sync.
• However for us we will focus on the most useful and commonly used features can get to the sync package(https://pkg.go.dev/sync).
• First let's discuss locks and Mutexes.
• The most common type related to locks in go is the locker interface type which encapsulates the functionality for locking and unlocking in go.
• So it has a lock method and the unlock method.
• So in order to enable the functionality of the locker interface, We can use a mutex struct type which will provide the ability to lock and unlock pieces of code between threads or to be more accurate code between Go routines.
• Locking is an important feature in any modern programming language.
• We need to avoid race conditions with multiple Go routines need to access the same memory space.

5.1-1.go.go

package main

import (
    "fmt"
    "sync"
)

type safeCounter struct {
    i int
    sync.Mutex
}

func main() {
    sc := new(safeCounter)

    for i := 0; i < 100; i++ {
        go sc.Increment()
        go sc.Decrement()
    }
    fmt.Println(sc.GetValue())
}

func (sc *safeCounter) Increment() {
    sc.Lock()
    sc.i++
    sc.Unlock()
}

func (sc *safeCounter) Decrement() {
    sc.Lock()
    sc.i--
    sc.Unlock()
}

func (sc *safeCounter) GetValue() int {
    return sc.i
}

• Let's explore some code here to learn how to use the Mutex type.
• So we'll start by defining the package as main in our code then we will import the fmt package and the sync package.
• So the sync package is a package we're learning then we'll create a new struct type called safeCounter
• safeCounter Basically it will become an object type which can increment an integer which is the integer here safely between Go routines.
• Meaning that when we use a safeCounter and we try to increment i
• It will ensure that no two go routines can increment i at the same time
• And we will use a mutex for that.
• Notice here that sync.Mutex type is added to the struct fields via embedding.
• And by doing this we can embed the functionality of sync.mutex directly to safeCounter.
• Now before we cover the main function let's look at the increment and the decrement methods for the safeCounter.
• So we will implement our methods on a pointer to safeCounter as shown.
• And the first thing we do before we either increment or decrement is to show a lock.
• So we will lock the mutex and because we use embedding, we can call the lock method of the mutex directly.
• So in case of the Increment, after during the lock, we will increment i and then we will unlock.
• In case of the Decrement, We will do the same except that we will decrement from i.
• Of course you must always unlock them mutex at the end to release the lock for other go routines to use.
• That shows us a simple example from mutex object could be used to protect a piece of code which is the incrementation of right here and the end.
• Let's implement that accessor in order to get the value of i.
• So the method will be called to get value and to return the value i.
• Now in the main function we'll start our code by initialising and new saveCounter pointer object by using the 'new' keyword and then we'll use a for loop to create a bunch of go routines to increments or decrement the integer concurrently.
• Integer here of course being like the field inside our safeCounter
• So for every single iteration for this for loop we will increment and the go routine and we will decrement on the other go routine and do continue doing that for 100 counts.
• Now at the end we will get the value of i by calling good value and will print it to the standard output.
• So this for loop will allow us to run numerouse Go routines at the same time.
• This will allow us to test the locks.
• Now it's important to know that this code is still susceptible race conditions and that is because we don't have a lock in GetValue method when we returning the value.
• So that means that GetValue could get called at the same time as when we were either incrementing or decrementing.
• So in order to test our code and see the effect of the locks, I will leave that GetValue method without a lock.
• And I will run the code with an important Go tool called -race which help us detect race conditions.
• So if I hit Enter we see a warning here of DATA RACE

PS C:\go\src\Udemy\lock> go run -race main.go
==================
WARNING: DATA RACE
Read at 0x00c0000120b0 by main goroutine:
  main.(*safeCounter).GetValue()
      C:/go/src/Udemy/lock/main.go:38 +0x134       
  main.main()
      C:/go/src/Udemy/lock/main.go:22 +0x140       

Previous write at 0x00c0000120b0 by goroutine 71:
  main.(*safeCounter).Increment()
      C:/go/src/Udemy/lock/main.go:27 +0x56        
  main.main쨌dwrap쨌1()
      C:/go/src/Udemy/lock/main.go:18 +0x39        

Goroutine 71 (finished) created at:
  main.main()
      C:/go/src/Udemy/lock/main.go:18 +0xc7        
==================
0
Found 1 data race(s)
exit status 66

• So that's what the -race flag can provide.
• It let us know of possible race conditions in our code.
• This is a very powerful tool to use when testing Go concurrent code.
• So if you look at the warning here
• You will find that it warns us about the fact that the code at line 22 which is where we were calling a GetValue without the lock could collide with the code at 27 which is when we're doing increment.
• So it looks like when we tried to run the code, the increment collided with the GetValue method.
• And Race condition race occurred now to see the effects of the locks, Let's do something like that.
• So I assign sc.i to a variable v and return v and then lock the piece of code that handles the value assignment.

Code GetValue modification

func (sc *safeCounter) GetValue() int {
    sc.Lock()
    v := sc.i
    sc.Unlock()
    return v
}

Result

1

• And we will run the code again with the dash race flag as before, this time we get no warnings.
• Let's try again. Same thing. No warning.
• Let's try a third time.
• No risk warning ever.
• And that is because the lock and unlock statements were placed here protected this piece of code against a risk condition that occurs when we try to retrieve i while it's being incremented or decremented.

func (*RWMutex) Lock

```
func (rw *RWMutex) Lock()
```

Lock locks rw for writing. If the lock is already locked for reading or writing, Lock blocks until the lock is available.

• Now let's discuss a more complex type of mutex which is read write mutex.
• This mutex allows two types of locks, Read Locks(RLock) and Write Locks(Lock).
• It allows go routines to read data concurrently but only write data via a single writer or a single go routine that does the right thing.
• So the read write mutex is simply named RWMutex when using RWMutex, the Lock method needs to be used when doing writes were as RLock methods need to be used when doing reads.
• the RWMute is more efficient than the standard mutex.
• If a large number of Freezes are expected from different Go routines that is because the Rlock method will not cause your Go routines to block if there are multiple reads.
• However when a write Lock engages, everything will lock including the other write locks as well as the RLocks.
• And that's how the RWMutex protects against risk conditions when it comes to executing writes on data.

5.1-2.go

package main

import (
    "fmt"
    "math/rand"
    "sync"
    "time"
)

type MapCounter struct {
    m map[int]int
    sync.RWMutex
}

func main() {
    mc := MapCounter{m: make(map[int]int)}
    go runWriter(&mc, 10)
    go runReaders(&mc, 10)
    go runReaders(&mc, 10)
    time.Sleep(15 * time.Second)
}

func runWriter(mc *MapCounter, n int) {
    for i := 0; i < n; i++ {
        mc.Lock()
        mc.m[i] = 1 * 10
        mc.Unlock()
        time.Sleep(1 * time.Second)
    }
}

func runReaders(mc *MapCounter, n int) {
    for {
        mc.RLock()
        v := mc.m[rand.Intn(n)]
        mc.RUnlock()
        fmt.Println(v)
        time.Sleep(1 * time.Second)
    }
}

https://go.dev/play/p/TBwwlIauNTv

• Very good use case for read write mutex is to protect reads and writes on maps.
• That is because map types are not thread safe by default in the Go language.
• So let's write a thread safe map using our new knowledge.
• So the design pattern in go to create a thread safe or go routine safe map is to create a struct type which will contain our map as well as the RWMutex object.
• Again here we use embedding to include the RWMutex object so that we can access the RWMutex methods directly from the MapCounter.
• Now our main function Let's create an object of type MapCounter where we will initialize the map field.
• So we use a struct literal initialization here to initialize the map field in the MapCounter struck and we stored this object in a variable called mc.
• Now before we explore the rest of the main function let's look at the writer and reader functions.
• So we'll start with a runWriter function which we'll write values to the map inside the for loop.
• We do that to test concurrency.
• This should actually be called Writers because we are doing multiple writers.
• So we'll create a for loop that will do n number of iterations and n is the second argument passed to the runWriter's function.
• The first arguments passed to the runWriters is a mc objects or pointers of a MapCounter object or thread safe or a Goroutine safe map.
• Then inside the For loop we will call the write Locks of the MapCounter.
• And this will be the RWMutex write lock.
  • Then we will assign a value to a key in the map.
• So because this is testing code, we will use some random values here.
• So we will use that value of i here like the number of our iteration as the key
• And then we will multiply i by 10 and the value that belongs to the i key.
• We then unlock after we're done with the write operation in our map and then we sleep for one second.
• So this is to simulate some delays in this function.
• We will write another function to simulate multiple readers trying to read a random value from the map.
• The function we called runReaders and it will take the pointer to the map as an argument as well as integer argument called n.
• And then for the readers you'll have an endless for loop where we will do RLock So Read Lock.
• On the map counter again this RLock comes from the embedded type RWMutex and then we do rand.Intn.
• So we use a random package for that.
• We use rand.Intn then we pass n and here should be the number of items in our map.
• So when we call rand.Intn and then pass and as an argument we are basically asking for values between [0] till [n-1]
• Then after we are done with the random read, we Unlock, we print out the random value retreat from the map.
• And then we sleep for a second again to simulate some additional work.
• You will notice here that we are using a pointer to the MapCounter object in the runReaders and runRiders functions.
• And the reason why we do that is to ensure we pass the MapCounter object here by reference.
• So In go passing by reference is the same as a pointer.
• So when we have the argument of type pointer then that allows us to pass the value by reference in our main code.
• The reason why we want to pass value by reference is because in go we need to use a reference in order to retain the original object and that is important because we want to run the writers and the readers on the same object.
• We don't want to pass copies of the object.
• We want the exact same object we defined here so that we can test concurrent code on the same piece of data.
• And our main function we will run three different Go routines
• One will be for the writers and the other two will be for the readers.
• We make the Readers run forever whereas Writers will run 10 times as we passed 10 here.
• Then we will use the sleep statement in order to prevent our program from exiting before the readers and the writers run for a little while.
• So this piece of code in effect we run a bunch of writers and a bunch of readers concurrently.
• Reading and Writing on the same object which is mc.
• Let's see that first.
• So let's open this code in a terminal in order to run it and let's use a race flag.
• We explored earlier to detect if there are any race conditions.
• So we see here that the races are going fine and the values are changing so writes are going as well.
• So everything seems good.
• Now what if I comment out the writelocks in our code.

func runWriter(mc *MapCounter, n int) { 
for i := 0; i < n; i++ { 
//mc.Lock()
mc.m[i] = 1 * 10 
//mc.Unlock() 
time.Sleep(1 * time.Second) } }

Result

==================  
WARNING: DATA RACE  
Read at 0x00c00004a048 by goroutine 8:  
main.runReaders()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:35 +0x96  
main.main쨌dwrap쨌2()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:18 +0x47

Previous write at 0x00c00004a048 by goroutine 7:  
main.runWriter()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:26 +0x69  
main.main쨌dwrap쨌1()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:17 +0x47

Goroutine 8 (running) created at:  
main.main()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:18 +0x198

Goroutine 7 (running) created at:  
main.main()  
C:/Users/David/go/src/udemy/udemy/lock/5.1-2.go:17 +0x118  
==================

• Now let's try to run this same command again with a dash race flag and see what we get and right away we get the warning.
• And that is because our writers are not protected anymore.
• A race condition can occur between a go routine writing to this object and another go routine reading from the object.
• And that shows us the power of the RWMutex.

Once

import (  
"fmt"  
"sync"  
)

func main() {  
var once sync.Once  
onceBody := func() {  
fmt.Println("Only once")  
}  
done := make(chan bool)  
for i := 0; i < 10; i++ {  
go func() {  
once.Do(onceBody)  
done <- true  
}()  
}  
for i := 0; i < 10; i++ {  
<-done  
}  
}

https://go.dev/play/p/rmPXf540Qof

• So another important construct that we need to cover is the once type.
• We have covered the once struct in the past when we wrote the Hydra logger.
• So when the once type is enabled, it allows us to run a piece of code only once over the lifetime of our application and prevent the peace of code from ever getting executed again.
• So the way to use once is as shown here and the example in the same package page.
• So we have it here we wrote a function called onceBody
• And then in order to run this function once.
• We just run once.Do
• so when we call once.DO, pass a function with that signature to the once.Do method.
• This function will only get executed once for the lifetime of our program.
• So if you look here we see here that the once.Do here gets called inside the go routine which gets called inside for loop of 10 counts.
• So even though this code exists inside the for loop of 10 counts, this function onceBody will only get executed once.
• So when I run this I should see only once once.
• So let's try that we see here we see it only once.
• However if I comment that out and just run onceBody like any regular function.

After

package main

import (  
"fmt"  
//"sync"  
)

func main() {  
//var once sync.Once  
onceBody := func() {  
fmt.Println("Only once")  
}  
done := make(chan bool)  
for i := 0; i < 10; i++ {  
go func() {  
//once.Do(onceBody)  
onceBody()  
done <- true  
}()  
}  
for i := 0; i < 10; i++ {  
<-done  
}  
}

Result

Only once  
Only once  
Only once  
Only once  
Only once  
Only once  
Only once  
Only once  
Only once  
Only once

• Hit run now, we see here only once repeated 10 times.

Hlogger

package hlogger

import (  
"log"  
"os"  
"sync"  
)

type hydraLogger struct {  
\*log.Logger  
filename string  
}

var hlogger \*hydraLogger  
var once sync.Once

//GetInstance creates a singleton instance of the hydra logger  
func GetInstance() \*hydraLogger {  
once.Do(func() {  
hlogger = createLogger("hydralogger.log")  
})  
return hlogger  
}

//Create a logger instance  
func createLogger(fname string) \*hydraLogger {  
file, \_ := os.OpenFile(fname, os.O\_RDWR|os.O\_CREATE|os.O\_TRUNC, 0777)

return &hydraLogger{
filename: fname,
Logger: log.New(file, "Hydra ", log.Lshortfile),
}

}

• So let's revisit the hlogger code and look at where we used once.Do
• as shown, We use it to ensure that the hydro logger object will only be created once through the lifetime of our application or micro ervice and we did that to ensure that a single logger will serve our entire application.
• So the way we activated once was the same as in the example
• First we created a variable of type sync.Once.
• So typically people call the variable once but of course we can call it over name we like.
• And then when we needed to use one object we just called .do method and we passed a function with a signature to the do method perfect.

WaitGroup

package main

import (  
"fmt"  
"math/rand"  
"sync"  
"time"  
)

func main() {  
  var wg sync.WaitGroup

  for i := 0; 1 <= 5; i++ {

//Increment the WaitGroup conter.
wg.Add(1)

    //Launch a goroutine
      go func(i int) {
      // Decrement the counter when the goroutine completes.
      defer wg.Done()
      // Do some work
      time.Sleep(time.Duration(rand.Intn(3)) * time.Second)
      fmt.Println("Work done for ", i)
      }(i)
    }

wg.Wait()
}

https://go.dev/play/p/v2amZNMERxn

• Another important type in the sync package is a WaitGroup.
• We use a WaitGroup to wait for a collection of go routines to finish.
• Let's see how we use it in our code.
• Let's see a code example where we use the WaitGroup.
• So this will be a test program so the package will be package main and we will import a bunch of packages here for our use.
• So all import fmt, Math/rand because we will try to obtain a random value, sync of course to get hold of the WaitGroup type and we will import the time package.
• Now let's start our main function.
• So we start by creating a variable called WG type sync.WaitGroup.
• So this will be our WaitGroup object which we will use in our code.
• Then we'll create a bunch of go routines inside for loop.
• So the for loop will count from 0 to 5.
• So all of those will be six counts and for each iteration we will call wg.Add.
• Add a method in the WaitGroup object which increases the WaitGroup counter by the number we pass as an argument.
• So that WaitGroup counter is basically a number that we can use to identify the number of Go routines that are waiting on the WaitGroup to use that counter, We need to add 1 to int whenever we are about to create a new Go routine.
• So when we do wg.Add(1) before our go routine, we incremented the WaitGroup counter by 1 Right before we called our go routine then inside our Go routine first thing we do is called defer wg.Done() were as the done method will decrement the cWaitGroup counter by 1.
• We can see here when we go to the WaitGroup Code and go standard package, we see here that done is nothing but calling add was negative 1.
• So just decrementing the group counter by one.
• So when we call done with a defer we ensure that DDone will get called as this function or this go routine is exiting.
• We will then simulate some work by using time to sleep to sleep for a random variation from 0 to 2 seconds.
• Then we print out the fact that work is done and then we close our for loop and then at the end, whenever we want to listen on a WaitGroup or actually Wait on a WaitGroup we call wg.Wait() or waitGroup.wait.
• So basically wait is a method and a WaitGroup object that blocks the calling go routine till the WaitGroup counter is zero.
• That means that our main function will block here till all the go routines called wg.Done and exit.
• So that's how simple WaitGroups are.
• Again you just call Add before you start here new go routine then you call wg.Done inside your go routine.
• You can either use defer to make sure it gets called as your Go routine exits or you can just manually right at the end of the go routine function.

Result

Work done for 8619  
Work done for 654  
Work done for 8760  
Work done for 115  
.  
.  
.

• So let's run this code.
• See how it appears
• Hit enter and it actually runs.
• So we see here that we get the message that work was done for each iteration.
• And at the end it exits.
• And notice here how our main function waited till every single Go routine was done.
• If we go back to the same package page and look at the WaitGroup example, you'll find here an example for real use case for WaitGroups.

WaitGroups example

package main

import (  
"sync"  
)

type httpPkg struct{}

func (httpPkg) Get(url string) {}

var http httpPkg

func main() {  
var wg sync.WaitGroup  
var urls = \[\]string{  
"[http://www.golang.org/"](http://www.golang.org/%22),  
"[http://www.google.com/"](http://www.google.com/%22),  
"[http://www.somestupidname.com/"](http://www.somestupidname.com/%22),  
}  
for \_, url := range urls {  
// Increment the WaitGroup counter.  
wg.Add(1)  
// Launch a goroutine to fetch the URL.  
go func(url string) {  
// Decrement the counter when the goroutine completes.  
defer wg.Done()  
// Fetch the URL.  
http.Get(url)  
}(url)  
}  
// Wait for all HTTP fetches to complete.  
wg.Wait()  
}

• In this example, we use WaitGroup to write performant web client that fetches a bunch of URL concurrently using a http.Get.
• This saves time and increases performance.
• Then at the end we block the calling go routine till all the URLs are processed concurrently.

Summary

• We learned how to sync our code between multiple go routines.
• This will be an additional tool in our arsenal to design powerful software and the Go language in the next
• We will talk about how to execute code in periodic time intervals using Go timers and tickers.