Preview Mode Links will not work in preview mode

Sep 16, 2019



Infosec Campout report


Jay Beale (co-lead for audit) *Bust-a-Kube*  

Aaron Small (product mgr at GKE/Google)


Atreides Partners

Trail of Bits


What was the Audit? 

How did it come about? 


Who were the players?

    Kubernetes Working Group

        Aaron, Craig, Jay, Joel

    Outside vendors:

        Atredis: Josh, Nathan Keltner

        Trail of Bits: Stefan Edwards, Bobby Tonic , Dominik

    Kubernetes Project Leads/Devs

        Interviewed devs -- this was much of the info that went into the threat model

        Rapid Risk Assessments - let’s put the GitHub repository in the show notes


What did it produce?

    Vuln Report

    Threat Model -

    White Papers


    Discuss the results:

        Threat model findings

            Controls silently fail, leading to a false sense of security

                Pod Security Policies, Egress Network Rules

            Audit model isn’t strong enough for non-repudiation

                By default, API server doesn’t log user movements through system

            TLS Encryption weaknesses

                Most components accept cleartext HTTP

                Boot strapping to add Kubelets is particularly weak       

                Multiple components do not check certificates and/or use self-signed certs

                HTTPS isn’t enforced

                Certificates are long-lived, with no revocation capability

                Etcd doesn’t authenticate connections by default

            Controllers all Bundled together

                Confused Deputy: b/c lower priv controllers bundled in same binary as higher

            Secrets not encrypted at rest by default

            Etcd doesn’t have signatures on its write-ahead log

            DoS attack: you can set anti-affinity on your pods to get nothing else scheduled on their nodes


            Port 10255 has an unauthenticated HTTP server for status and health checking


        Vulns / Findings (not complete list, but interesting)

            Hostpath pod security policy bypass via persistent volumes

            TOCTOU when moving PID to manager’s group

            Improperly patched directory traversal in kubectl cp

            Bearer tokens revealed in logs

            Lots of MitM risk:

            SSH not checking fingerprints: InsecureIgnoreHostKey

            gRPC transport seems all set to WithInsecure()

HTTPS connections not checking certs 

            Some HTTPS connections are unauthenticated

            Output encoding on JSON construction

                This might lead to further work, as JSON can get written to logs that may be consumed elsewhere.

            Non-constant time check on passwords

Lack of re-use / library-ification of code


    Who will use these findings and how? Devs, google, bad guys? 

    Any new audit tools created from this? 


Brad geesaman “Hacking and Hardening Kubernetes Clusters by Example [I] - Brad Geesaman, Symantec


Aaron Small: 







Scope for testing:

        Source code review (what languages did they have to review?)

            Golang, shell, ...


Networking (discuss the networking *internal* *external*

Cryptography (TLS, data stores)


RBAC (which roles were tested? Just admin/non-admin *best practice is no admin/least priv*)


Namespace traversals

Namespace claims




Setup a bunch of environments?

    Primarily set up a single environment IIRC

    Combination of code audit and active ?fuzzing?

        What does one fuzz on a K8s environment?

Tested with latest alpha or production versions?

    Version 1.13 or 1.14 - version locked at whatever was current - K8S releases a new version every 3 months, so this is a challenge and means we have to keep auditing.

Tested mulitple different types of k8s implementations?

    Tested primarily against kubespray (


Bug Bounty program:


Check out our Store on Teepub!

Join us on our #Slack Channel! Send a request to @brakesec on Twitter or email

#Brakesec Store!:



#Youtube Channel:

#iTunes Store Link:

#Google Play Store:

Our main site:

#iHeartRadio App:


Comments, Questions, Feedback:

Support Brakeing Down Security Podcast by using our #Paypal OR our #Patreon

#Twitter@brakesec @boettcherpwned @bryanbrake @infosystir

#Player.FM :

#Stitcher Network:

#TuneIn Radio App: