rebase: update replaced k8s.io modules to v0.33.0

Signed-off-by: Niels de Vos <ndevos@ibm.com>
This commit is contained in:
Niels de Vos
2025-05-07 13:13:33 +02:00
committed by mergify[bot]
parent dd77e72800
commit 107407b44b
1723 changed files with 65035 additions and 175239 deletions

View File

@ -6,6 +6,80 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
## [Unreleased] ##
## [0.4.1] - 2025-01-28 ##
### Fixed ###
- The restrictions added for `root` paths passed to `SecureJoin` in 0.4.0 was
found to be too strict and caused some regressions when folks tried to
update, so this restriction has been relaxed to only return an error if the
path contains a `..` component. We still recommend users use `filepath.Clean`
(and even `filepath.EvalSymlinks`) on the `root` path they are using, but at
least you will no longer be punished for "trivial" unclean paths.
## [0.4.0] - 2025-01-13 ##
### Breaking ####
- `SecureJoin(VFS)` will now return an error if the provided `root` is not a
`filepath.Clean`'d path.
While it is ultimately the responsibility of the caller to ensure the root is
a safe path to use, passing a path like `/symlink/..` as a root would result
in the `SecureJoin`'d path being placed in `/` even though `/symlink/..`
might be a different directory, and so we should more strongly discourage
such usage.
All major users of `securejoin.SecureJoin` already ensure that the paths they
provide are safe (and this is ultimately a question of user error), but
removing this foot-gun is probably a good idea. Of course, this is
necessarily a breaking API change (though we expect no real users to be
affected by it).
Thanks to [Erik Sjölund](https://github.com/eriksjolund), who initially
reported this issue as a possible security issue.
- `MkdirAll` and `MkdirHandle` now take an `os.FileMode`-style mode argument
instead of a raw `unix.S_*`-style mode argument, which may cause compile-time
type errors depending on how you use `filepath-securejoin`. For most users,
there will be no change in behaviour aside from the type change (as the
bottom `0o777` bits are the same in both formats, and most users are probably
only using those bits).
However, if you were using `unix.S_ISVTX` to set the sticky bit with
`MkdirAll(Handle)` you will need to switch to `os.ModeSticky` otherwise you
will get a runtime error with this update. In addition, the error message you
will get from passing `unix.S_ISUID` and `unix.S_ISGID` will be different as
they are treated as invalid bits now (note that previously passing said bits
was also an error).
## [0.3.6] - 2024-12-17 ##
### Compatibility ###
- The minimum Go version requirement for `filepath-securejoin` is now Go 1.18
(we use generics internally).
For reference, `filepath-securejoin@v0.3.0` somewhat-arbitrarily bumped the
Go version requirement to 1.21.
While we did make some use of Go 1.21 stdlib features (and in principle Go
versions <= 1.21 are no longer even supported by upstream anymore), some
downstreams have complained that the version bump has meant that they have to
do workarounds when backporting fixes that use the new `filepath-securejoin`
API onto old branches. This is not an ideal situation, but since using this
library is probably better for most downstreams than a hand-rolled
workaround, we now have compatibility shims that allow us to build on older
Go versions.
- Lower minimum version requirement for `golang.org/x/sys` to `v0.18.0` (we
need the wrappers for `fsconfig(2)`), which should also make backporting
patches to older branches easier.
## [0.3.5] - 2024-12-06 ##
### Fixed ###
- `MkdirAll` will now no longer return an `EEXIST` error if two racing
processes are creating the same directory. We will still verify that the path
is a directory, but this will avoid spurious errors when multiple threads or
programs are trying to `MkdirAll` the same path. opencontainers/runc#4543
## [0.3.4] - 2024-10-09 ##
### Fixed ###
@ -164,8 +238,12 @@ This is our first release of `github.com/cyphar/filepath-securejoin`,
containing a full implementation with a coverage of 93.5% (the only missing
cases are the error cases, which are hard to mocktest at the moment).
[Unreleased]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.4...HEAD
[0.3.3]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.3...v0.3.4
[Unreleased]: https://github.com/cyphar/filepath-securejoin/compare/v0.4.1...HEAD
[0.4.1]: https://github.com/cyphar/filepath-securejoin/compare/v0.4.0...v0.4.1
[0.4.0]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.6...v0.4.0
[0.3.6]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.5...v0.3.6
[0.3.5]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.4...v0.3.5
[0.3.4]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.3...v0.3.4
[0.3.3]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.2...v0.3.3
[0.3.2]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.1...v0.3.2
[0.3.1]: https://github.com/cyphar/filepath-securejoin/compare/v0.3.0...v0.3.1

View File

@ -1 +1 @@
0.3.4
0.4.1

View File

@ -0,0 +1,18 @@
//go:build linux && go1.20
// Copyright (C) 2024 SUSE LLC. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package securejoin
import (
"fmt"
)
// wrapBaseError is a helper that is equivalent to fmt.Errorf("%w: %w"), except
// that on pre-1.20 Go versions only errors.Is() works properly (errors.Unwrap)
// is only guaranteed to give you baseErr.
func wrapBaseError(baseErr, extraErr error) error {
return fmt.Errorf("%w: %w", extraErr, baseErr)
}

View File

@ -0,0 +1,38 @@
//go:build linux && !go1.20
// Copyright (C) 2024 SUSE LLC. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package securejoin
import (
"fmt"
)
type wrappedError struct {
inner error
isError error
}
func (err wrappedError) Is(target error) bool {
return err.isError == target
}
func (err wrappedError) Unwrap() error {
return err.inner
}
func (err wrappedError) Error() string {
return fmt.Sprintf("%v: %v", err.isError, err.inner)
}
// wrapBaseError is a helper that is equivalent to fmt.Errorf("%w: %w"), except
// that on pre-1.20 Go versions only errors.Is() works properly (errors.Unwrap)
// is only guaranteed to give you baseErr.
func wrapBaseError(baseErr, extraErr error) error {
return wrappedError{
inner: baseErr,
isError: extraErr,
}
}

View File

@ -0,0 +1,32 @@
//go:build linux && go1.21
// Copyright (C) 2024 SUSE LLC. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package securejoin
import (
"slices"
"sync"
)
func slices_DeleteFunc[S ~[]E, E any](slice S, delFn func(E) bool) S {
return slices.DeleteFunc(slice, delFn)
}
func slices_Contains[S ~[]E, E comparable](slice S, val E) bool {
return slices.Contains(slice, val)
}
func slices_Clone[S ~[]E, E any](slice S) S {
return slices.Clone(slice)
}
func sync_OnceValue[T any](f func() T) func() T {
return sync.OnceValue(f)
}
func sync_OnceValues[T1, T2 any](f func() (T1, T2)) func() (T1, T2) {
return sync.OnceValues(f)
}

View File

@ -0,0 +1,124 @@
//go:build linux && !go1.21
// Copyright (C) 2024 SUSE LLC. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package securejoin
import (
"sync"
)
// These are very minimal implementations of functions that appear in Go 1.21's
// stdlib, included so that we can build on older Go versions. Most are
// borrowed directly from the stdlib, and a few are modified to be "obviously
// correct" without needing to copy too many other helpers.
// clearSlice is equivalent to the builtin clear from Go 1.21.
// Copied from the Go 1.24 stdlib implementation.
func clearSlice[S ~[]E, E any](slice S) {
var zero E
for i := range slice {
slice[i] = zero
}
}
// Copied from the Go 1.24 stdlib implementation.
func slices_IndexFunc[S ~[]E, E any](s S, f func(E) bool) int {
for i := range s {
if f(s[i]) {
return i
}
}
return -1
}
// Copied from the Go 1.24 stdlib implementation.
func slices_DeleteFunc[S ~[]E, E any](s S, del func(E) bool) S {
i := slices_IndexFunc(s, del)
if i == -1 {
return s
}
// Don't start copying elements until we find one to delete.
for j := i + 1; j < len(s); j++ {
if v := s[j]; !del(v) {
s[i] = v
i++
}
}
clearSlice(s[i:]) // zero/nil out the obsolete elements, for GC
return s[:i]
}
// Similar to the stdlib slices.Contains, except that we don't have
// slices.Index so we need to use slices.IndexFunc for this non-Func helper.
func slices_Contains[S ~[]E, E comparable](s S, v E) bool {
return slices_IndexFunc(s, func(e E) bool { return e == v }) >= 0
}
// Copied from the Go 1.24 stdlib implementation.
func slices_Clone[S ~[]E, E any](s S) S {
// Preserve nil in case it matters.
if s == nil {
return nil
}
return append(S([]E{}), s...)
}
// Copied from the Go 1.24 stdlib implementation.
func sync_OnceValue[T any](f func() T) func() T {
var (
once sync.Once
valid bool
p any
result T
)
g := func() {
defer func() {
p = recover()
if !valid {
panic(p)
}
}()
result = f()
f = nil
valid = true
}
return func() T {
once.Do(g)
if !valid {
panic(p)
}
return result
}
}
// Copied from the Go 1.24 stdlib implementation.
func sync_OnceValues[T1, T2 any](f func() (T1, T2)) func() (T1, T2) {
var (
once sync.Once
valid bool
p any
r1 T1
r2 T2
)
g := func() {
defer func() {
p = recover()
if !valid {
panic(p)
}
}()
r1, r2 = f()
f = nil
valid = true
}
return func() (T1, T2) {
once.Do(g)
if !valid {
panic(p)
}
return r1, r2
}
}

View File

@ -1,5 +1,5 @@
// Copyright (C) 2014-2015 Docker Inc & Go Authors. All rights reserved.
// Copyright (C) 2017-2024 SUSE LLC. All rights reserved.
// Copyright (C) 2017-2025 SUSE LLC. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
@ -24,6 +24,31 @@ func IsNotExist(err error) bool {
return errors.Is(err, os.ErrNotExist) || errors.Is(err, syscall.ENOTDIR) || errors.Is(err, syscall.ENOENT)
}
// errUnsafeRoot is returned if the user provides SecureJoinVFS with a path
// that contains ".." components.
var errUnsafeRoot = errors.New("root path provided to SecureJoin contains '..' components")
// stripVolume just gets rid of the Windows volume included in a path. Based on
// some godbolt tests, the Go compiler is smart enough to make this a no-op on
// Linux.
func stripVolume(path string) string {
return path[len(filepath.VolumeName(path)):]
}
// hasDotDot checks if the path contains ".." components in a platform-agnostic
// way.
func hasDotDot(path string) bool {
// If we are on Windows, strip any volume letters. It turns out that
// C:..\foo may (or may not) be a valid pathname and we need to handle that
// leading "..".
path = stripVolume(path)
// Look for "/../" in the path, but we need to handle leading and trailing
// ".."s by adding separators. Doing this with filepath.Separator is ugly
// so just convert to Unix-style "/" first.
path = filepath.ToSlash(path)
return strings.Contains("/"+path+"/", "/../")
}
// SecureJoinVFS joins the two given path components (similar to [filepath.Join]) except
// that the returned path is guaranteed to be scoped inside the provided root
// path (when evaluated). Any symbolic links in the path are evaluated with the
@ -46,7 +71,22 @@ func IsNotExist(err error) bool {
// provided via direct input or when evaluating symlinks. Therefore:
//
// "C:\Temp" + "D:\path\to\file.txt" results in "C:\Temp\path\to\file.txt"
//
// If the provided root is not [filepath.Clean] then an error will be returned,
// as such root paths are bordering on somewhat unsafe and using such paths is
// not best practice. We also strongly suggest that any root path is first
// fully resolved using [filepath.EvalSymlinks] or otherwise constructed to
// avoid containing symlink components. Of course, the root also *must not* be
// attacker-controlled.
func SecureJoinVFS(root, unsafePath string, vfs VFS) (string, error) {
// The root path must not contain ".." components, otherwise when we join
// the subpath we will end up with a weird path. We could work around this
// in other ways but users shouldn't be giving us non-lexical root paths in
// the first place.
if hasDotDot(root) {
return "", errUnsafeRoot
}
// Use the os.* VFS implementation if none was specified.
if vfs == nil {
vfs = osVFS{}
@ -59,9 +99,10 @@ func SecureJoinVFS(root, unsafePath string, vfs VFS) (string, error) {
linksWalked int
)
for remainingPath != "" {
if v := filepath.VolumeName(remainingPath); v != "" {
remainingPath = remainingPath[len(v):]
}
// On Windows, if we managed to end up at a path referencing a volume,
// drop the volume to make sure we don't end up with broken paths or
// escaping the root volume.
remainingPath = stripVolume(remainingPath)
// Get the next path component.
var part string

View File

@ -12,7 +12,6 @@ import (
"os"
"path"
"path/filepath"
"slices"
"strings"
"golang.org/x/sys/unix"
@ -113,7 +112,7 @@ func (s *symlinkStack) push(dir *os.File, remainingPath, linkTarget string) erro
return nil
}
// Split the link target and clean up any "" parts.
linkTargetParts := slices.DeleteFunc(
linkTargetParts := slices_DeleteFunc(
strings.Split(linkTarget, "/"),
func(part string) bool { return part == "" || part == "." })

View File

@ -11,7 +11,6 @@ import (
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"golang.org/x/sys/unix"
@ -22,6 +21,33 @@ var (
errPossibleAttack = errors.New("possible attack detected")
)
// modePermExt is like os.ModePerm except that it also includes the set[ug]id
// and sticky bits.
const modePermExt = os.ModePerm | os.ModeSetuid | os.ModeSetgid | os.ModeSticky
//nolint:cyclop // this function needs to handle a lot of cases
func toUnixMode(mode os.FileMode) (uint32, error) {
sysMode := uint32(mode.Perm())
if mode&os.ModeSetuid != 0 {
sysMode |= unix.S_ISUID
}
if mode&os.ModeSetgid != 0 {
sysMode |= unix.S_ISGID
}
if mode&os.ModeSticky != 0 {
sysMode |= unix.S_ISVTX
}
// We don't allow file type bits.
if mode&os.ModeType != 0 {
return 0, fmt.Errorf("%w %+.3o (%s): type bits not permitted", errInvalidMode, mode, mode)
}
// We don't allow other unknown modes.
if mode&^modePermExt != 0 || sysMode&unix.S_IFMT != 0 {
return 0, fmt.Errorf("%w %+.3o (%s): unknown mode bits", errInvalidMode, mode, mode)
}
return sysMode, nil
}
// MkdirAllHandle is equivalent to [MkdirAll], except that it is safer to use
// in two respects:
//
@ -40,17 +66,17 @@ var (
// a brand new lookup of unsafePath (such as with [SecureJoin] or openat2) after
// doing [MkdirAll]. If you intend to open the directory after creating it, you
// should use MkdirAllHandle.
func MkdirAllHandle(root *os.File, unsafePath string, mode int) (_ *os.File, Err error) {
// Make sure there are no os.FileMode bits set.
if mode&^0o7777 != 0 {
return nil, fmt.Errorf("%w for mkdir 0o%.3o", errInvalidMode, mode)
func MkdirAllHandle(root *os.File, unsafePath string, mode os.FileMode) (_ *os.File, Err error) {
unixMode, err := toUnixMode(mode)
if err != nil {
return nil, err
}
// On Linux, mkdirat(2) (and os.Mkdir) silently ignore the suid and sgid
// bits. We could also silently ignore them but since we have very few
// users it seems more prudent to return an error so users notice that
// these bits will not be set.
if mode&^0o1777 != 0 {
return nil, fmt.Errorf("%w for mkdir 0o%.3o: suid and sgid are ignored by mkdir", errInvalidMode, mode)
if unixMode&^0o1777 != 0 {
return nil, fmt.Errorf("%w for mkdir %+.3o: suid and sgid are ignored by mkdir", errInvalidMode, mode)
}
// Try to open as much of the path as possible.
@ -93,7 +119,7 @@ func MkdirAllHandle(root *os.File, unsafePath string, mode int) (_ *os.File, Err
}
remainingParts := strings.Split(remainingPath, string(filepath.Separator))
if slices.Contains(remainingParts, "..") {
if slices_Contains(remainingParts, "..") {
// The path contained ".." components after the end of the "real"
// components. We could try to safely resolve ".." here but that would
// add a bunch of extra logic for something that it's not clear even
@ -105,9 +131,6 @@ func MkdirAllHandle(root *os.File, unsafePath string, mode int) (_ *os.File, Err
return nil, fmt.Errorf("%w: yet-to-be-created path %q contains '..' components", unix.ENOENT, remainingPath)
}
// Make sure the mode doesn't have any type bits.
mode &^= unix.S_IFMT
// Create the remaining components.
for _, part := range remainingParts {
switch part {
@ -119,11 +142,20 @@ func MkdirAllHandle(root *os.File, unsafePath string, mode int) (_ *os.File, Err
// NOTE: mkdir(2) will not follow trailing symlinks, so we can safely
// create the final component without worrying about symlink-exchange
// attacks.
if err := unix.Mkdirat(int(currentDir.Fd()), part, uint32(mode)); err != nil {
//
// If we get -EEXIST, it's possible that another program created the
// directory at the same time as us. In that case, just continue on as
// if we created it (if the created inode is not a directory, the
// following open call will fail).
if err := unix.Mkdirat(int(currentDir.Fd()), part, unixMode); err != nil && !errors.Is(err, unix.EEXIST) {
err = &os.PathError{Op: "mkdirat", Path: currentDir.Name() + "/" + part, Err: err}
// Make the error a bit nicer if the directory is dead.
if err2 := isDeadInode(currentDir); err2 != nil {
err = fmt.Errorf("%w (%w)", err, err2)
if deadErr := isDeadInode(currentDir); deadErr != nil {
// TODO: Once we bump the minimum Go version to 1.20, we can use
// multiple %w verbs for this wrapping. For now we need to use a
// compatibility shim for older Go versions.
//err = fmt.Errorf("%w (%w)", err, deadErr)
err = wrapBaseError(err, deadErr)
}
return nil, err
}
@ -188,10 +220,7 @@ func MkdirAllHandle(root *os.File, unsafePath string, mode int) (_ *os.File, Err
// If you plan to open the directory after you have created it or want to use
// an open directory handle as the root, you should use [MkdirAllHandle] instead.
// This function is a wrapper around [MkdirAllHandle].
//
// NOTE: The mode argument must be set the unix mode bits (unix.S_I...), not
// the Go generic mode bits ([os.FileMode]...).
func MkdirAll(root, unsafePath string, mode int) error {
func MkdirAll(root, unsafePath string, mode os.FileMode) error {
rootDir, err := os.OpenFile(root, unix.O_PATH|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return err

View File

@ -12,12 +12,11 @@ import (
"os"
"path/filepath"
"strings"
"sync"
"golang.org/x/sys/unix"
)
var hasOpenat2 = sync.OnceValue(func() bool {
var hasOpenat2 = sync_OnceValue(func() bool {
fd, err := unix.Openat2(unix.AT_FDCWD, ".", &unix.OpenHow{
Flags: unix.O_PATH | unix.O_CLOEXEC,
Resolve: unix.RESOLVE_NO_SYMLINKS | unix.RESOLVE_IN_ROOT,

View File

@ -12,7 +12,6 @@ import (
"os"
"runtime"
"strconv"
"sync"
"golang.org/x/sys/unix"
)
@ -54,7 +53,7 @@ func verifyProcRoot(procRoot *os.File) error {
return nil
}
var hasNewMountApi = sync.OnceValue(func() bool {
var hasNewMountApi = sync_OnceValue(func() bool {
// All of the pieces of the new mount API we use (fsopen, fsconfig,
// fsmount, open_tree) were added together in Linux 5.1[1,2], so we can
// just check for one of the syscalls and the others should also be
@ -192,11 +191,11 @@ func doGetProcRoot() (*os.File, error) {
return procRoot, err
}
var getProcRoot = sync.OnceValues(func() (*os.File, error) {
var getProcRoot = sync_OnceValues(func() (*os.File, error) {
return doGetProcRoot()
})
var hasProcThreadSelf = sync.OnceValue(func() bool {
var hasProcThreadSelf = sync_OnceValue(func() bool {
return unix.Access("/proc/thread-self/", unix.F_OK) == nil
})
@ -265,12 +264,20 @@ func procThreadSelf(procRoot *os.File, subpath string) (_ *os.File, _ procThread
Resolve: unix.RESOLVE_BENEATH | unix.RESOLVE_NO_XDEV | unix.RESOLVE_NO_MAGICLINKS,
})
if err != nil {
return nil, nil, fmt.Errorf("%w: %w", errUnsafeProcfs, err)
// TODO: Once we bump the minimum Go version to 1.20, we can use
// multiple %w verbs for this wrapping. For now we need to use a
// compatibility shim for older Go versions.
//err = fmt.Errorf("%w: %w", errUnsafeProcfs, err)
return nil, nil, wrapBaseError(err, errUnsafeProcfs)
}
} else {
handle, err = openatFile(procRoot, threadSelf+subpath, unix.O_PATH|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0)
if err != nil {
return nil, nil, fmt.Errorf("%w: %w", errUnsafeProcfs, err)
// TODO: Once we bump the minimum Go version to 1.20, we can use
// multiple %w verbs for this wrapping. For now we need to use a
// compatibility shim for older Go versions.
//err = fmt.Errorf("%w: %w", errUnsafeProcfs, err)
return nil, nil, wrapBaseError(err, errUnsafeProcfs)
}
defer func() {
if Err != nil {
@ -289,12 +296,17 @@ func procThreadSelf(procRoot *os.File, subpath string) (_ *os.File, _ procThread
return handle, runtime.UnlockOSThread, nil
}
var hasStatxMountId = sync.OnceValue(func() bool {
// STATX_MNT_ID_UNIQUE is provided in golang.org/x/sys@v0.20.0, but in order to
// avoid bumping the requirement for a single constant we can just define it
// ourselves.
const STATX_MNT_ID_UNIQUE = 0x4000
var hasStatxMountId = sync_OnceValue(func() bool {
var (
stx unix.Statx_t
// We don't care which mount ID we get. The kernel will give us the
// unique one if it is supported.
wantStxMask uint32 = unix.STATX_MNT_ID_UNIQUE | unix.STATX_MNT_ID
wantStxMask uint32 = STATX_MNT_ID_UNIQUE | unix.STATX_MNT_ID
)
err := unix.Statx(-int(unix.EBADF), "/", 0, int(wantStxMask), &stx)
return err == nil && stx.Mask&wantStxMask != 0
@ -310,7 +322,7 @@ func getMountId(dir *os.File, path string) (uint64, error) {
stx unix.Statx_t
// We don't care which mount ID we get. The kernel will give us the
// unique one if it is supported.
wantStxMask uint32 = unix.STATX_MNT_ID_UNIQUE | unix.STATX_MNT_ID
wantStxMask uint32 = STATX_MNT_ID_UNIQUE | unix.STATX_MNT_ID
)
err := unix.Statx(int(dir.Fd()), path, unix.AT_EMPTY_PATH|unix.AT_SYMLINK_NOFOLLOW, int(wantStxMask), &stx)