Home Posts About Contact

Cloud Application Development

Containerize applications and run them in the cloud with OpenShift

abstract sockets

Abstract sockets are a great way for multiple containers in the same OpenShift or Kubernetes pod to communicate with each other, without using any additional storage volumes or network configuration. Since all the containers in a single OpenShift and Kubernetes pod will share the same IPC namespace by default, you can reference a Unix Domain Socket in your program like '@my-abstract-socket', without having to create and keep track of mounth paths.  


More about abstract sockets

Abstract sockets on Linux function the same way as other Unix Domain Sockets, except they don't have a filepath associated with them. Instead, you access them via the '@' symbol. For example, /var/run/crio/crio.sock is the socket file associated with the cri-o Container Runtime Interface, and we would access it from our own programs using that file path.

Wheras we might create an abstract socket called 'example.sock' and access it by referring to '@example.sock'; No file path needed. Note that it doesn't require a '.sock' extension, that's just to keep up with naming conventions.

Use cases

They are especially useful for container to container communication within the same pod.

The following is a list of some advantages over traditional Unix Domain Sockets in a containerized setting, and some of these are relevant to non-containerized workloads as well:

Other considerations

When wouldn't we want to use abstract sockets?

Examples

Some practical RPC client and server examples in Golang first, with brief Python examples and additional details to follow: Golang - Listening on an abstract socket
package main

import (
	"fmt"
	"log"
	"net"
	"net/http"
	"net/rpc"
	"os"
)

// RPCSrv is the base type that needs to be exported for RPC to work.
type RPCSrv struct {
}

// MyRPCExportedFunction is the RPC-exported method that returns a bytestring
// to an imaginary container in the same pod.
// The reply can be anything, really.
// I typically use Go structs for more complex, structured data, and send it as marshalled json.
// Here we're just appending a string and sending it as bytes manually.
func (g RPCSrv) MyRPCExportedFunction(input *string, reply *[]byte) error {
	replySuffix := []byte(" And a reply from the server.")

	*reply = append([]byte(*input), replySuffix[:]...)

	fmt.Println("RPC reply result was:", string((*reply)[:]))
	return nil
}

func main() {
	sock := "@my-abstract-socket.sock"

	// Start by cleaning up any leftover sockets of the same name.
	if err := os.RemoveAll(sock); err != nil {
		log.Fatal(err)
	}

	RPCSrv := new(RPCSrv)

	rpc.Register(RPCSrv)
	rpc.HandleHTTP()

	l, e := net.Listen("unix", sock)
	if e != nil {
		fmt.Println("Error starting listener:", e)
	}

	fmt.Println("Starting container info server with address:", sock)
	http.Serve(l, nil)
}
Golang - Connecting to an abstract socket
package main

import (
	"fmt"
	"net/rpc"
)

var (
	// data can be a Go datatype like a stuct, whatever the RPC server accepts.
	data = "Some values from the client,"
	// reply holds the return value of the RPC function
	reply []byte
	// remoteFunction has to match the format of BaseType.ExportedFunction, as defined in the server code.
	remoteFunction = "RPCSrv.MyRPCExportedFunction"
	// socketName is the name of the socket to which we will connect.
	// Note that preceding this name with '@' means we're connecting to a UDS in the abstract socket
	// namespace, and not a file on disk with that literal name.
	// Must match the name defined in the server code.
	socketName = "@my-abstract-socket.sock"
)

func main() {
	//fmt.Println(string(data[:]))
	fmt.Println("sending this data from client:", data)

	//client, err := rpc.DialHTTP("unix", "/tmp/my-abstract-socket.sock")
	client, err := rpc.DialHTTP("unix", "@my-abstract-socket.sock")
	if err != nil {
		fmt.Println("Error dialing socket:", err)
	}

	err = client.Call(remoteFunction, &data, &reply)
	if err != nil {
		fmt.Println("Error calling the RPC server's function:", err)
	}

	if len(reply) > 0 {
		fmt.Println("A reply was returned")
	} else {
		fmt.Println("The reply was empty")
	}
	fmt.Println(string(reply[:]))
}
Python - Listening on an abstract socket
import socket

sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

server_address = (hostname, int(hostport))

sock.bind('\0my-abstract-socket')

sock.listen(1)
Python - Connecting to an abstract socket


Python - Listing existing sockets
import psutil

# Abstract socket psutil output

mylist = psutil.net_connections(kind='unix')

print(mylist)

  [sconn(fd=3, family=, type=1, laddr='@my-abstract-socket', raddr=None, status='NONE', pid=10974)]



# TCP socket psutil output

inetlist = psutil.net_connections(kind='inet')

print(inetlist)

  [sconn(fd=3, family=, type=, laddr=('127.0.0.1', 8080), raddr=(), status='LISTEN', pid=10982)]
Listing sockets with lsof
lsof -U | grep 'my-abstract-socket\|COMMAND'

COMMAND      PID     USER    FD  TYPE             DEVICE SIZE/OFF          NODE                    NAME
python    129678 username    4u  unix 0x00000000341f0a92      0t0       1052783 @MyBindName type=STREAM