MFC the new fusefs driver

MFC r350665, r350990, r350992, r351039, r351042, r351061, r351066, r351113, r351560, r351961, r351963, r352021, r352025, r352230

r350665:
fusefs: merge from projects/fuse2

This commit imports the new fusefs driver. It raises the protocol level
from 7.8 to 7.23, fixes many bugs, adds a test suite for the driver, and
adds many new features. New features include:

* Optional kernel-side permissions checks (-o default_permissions)
* Implement VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK
* Allow interrupting FUSE operations
* Support named pipes and unix-domain sockets in fusefs file systems
* Forward UTIME_NOW during utimensat(2) to the daemon
* kqueue support for /dev/fuse
* Allow updating mounts with "mount -u"
* Allow exporting fusefs file systems over NFS
* Server-initiated invalidation of the name cache or data cache
* Respect RLIMIT_FSIZE
* Try to support servers as old as protocol 7.4

Performance enhancements include:

* Implement FUSE's FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags
* Cache file attributes
* Cache lookup entries, both positive and negative
* Server-selectable cache modes: writethrough, writeback, or uncached
* Write clustering
* Readahead
* Use counter(9) for statistical reporting

PR:		199934 216391 233783 234581 235773 235774 235775
PR:		236226 236231 236236 236291 236329 236381 236405
PR:		236327 236466 236472 236473 236474 236530 236557
PR:		236560 236844 237052 237181 237588 238565
Reviewed by:	bcr (man pages)
Reviewed by:	cem, ngie, rpokala, glebius, kib, bde, emaste (post-commit
		review on project branch)
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Pull Request:	https://reviews.freebsd.org/D21110

r350990:
fusefs: add SVN Keywords to the test files

Reported by:	SVN pre-commit hooks
MFC-With:	r350665
Sponsored by:	The FreeBSD Foundation

r350992:
fusefs: skip some tests when unsafe aio is disabled

MFC-With:       r350665
Sponsored by:   The FreeBSD Foundation

r351039:
fusefs: fix intermittency in the default_permissions.Unlink.ok test

The test needs to expect a FUSE_FORGET operation. Most of the time the test
would pass anyway, because by chance FUSE_FORGET would arrive after the
unmount.

MFC-With:	350665
Sponsored by:	The FreeBSD Foundation

r351042:
fusefs: Fix the size of fuse_getattr_in

In FUSE protocol 7.9, the size of the FUSE_GETATTR request has increased.
However, the fusefs driver is currently not sending the additional fields.
In our implementation, the additional fields are always zero, so I there
haven't been any test failures until now.  But fusefs-lkl requires the
request's length to be correct.

Fix this bug, and also enhance the test suite to catch similar bugs.

PR:		239830
MFC-With:	350665
Sponsored by:	The FreeBSD Foundation

r351061:
fusefs: fix the 32-bit build after 351042

Reported by:	jhb
MFC-With:	351042
Sponsored by:	The FreeBSD Foundation

r351066:
fusefs: fix conditional from r351061

The entirety of r351061 was a copy/paste error.  I'm sorry I've been
comitting so hastily.

Reported by:	rpokala
Reviewed by:	rpokala
MFC-With:	351061
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21265

r351113:
fusefs: don't send the namespace during listextattr

The FUSE_LISTXATTR operation always returns the full list of a file's
extended attributes, in all namespaces. There's no way to filter the list
server-side. However, currently FreeBSD's fusefs driver sends a namespace
string with the FUSE_LISTXATTR request. That behavior was probably copied
from fuse_vnop_getextattr, which has an attribute name argument. It's
been there ever since extended attribute support was added in r324620. This
commit removes it.

Reviewed by:	cem
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21280

r351560:
fusefs: Fix some bugs regarding the size of the LISTXATTR list

* A small error in r338152 let to the returned size always being exactly
  eight bytes too large.

* The FUSE_LISTXATTR operation works like Linux's listxattr(2): if the
  caller does not provide enough space, then the server should return ERANGE
  rather than return a truncated list.  That's true even though in FUSE's
  case the kernel doesn't provide space to the client at all; it simply
  requests a maximum size for the list.  We previously weren't handling the
  case where the server returns ERANGE even though the kernel requested as
  much size as the server had told us it needs; that can happen due to a
  race.

* We also need to ensure that a pathological server that always returns
  ERANGE no matter what size we request in FUSE_LISTXATTR won't cause an
  infinite loop in the kernel.  As of this commit, it will instead cause an
  infinite loop that exits and enters the kernel on each iteration, allowing
  signals to be processed.

Reviewed by:	cem
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21287

r351961:
Coverity fixes in fusefs(5)

CID 1404532 fixes a signed vs unsigned comparison error in fuse_vnop_bmap.
It could potentially have resulted in VOP_BMAP reporting too many
consecutive blocks.

CID 1404364 is much worse. It was an array access by an untrusted,
user-provided variable. It could potentially have resulted in a malicious
file system crashing the kernel or worse.

Reported by:	Coverity
Reviewed by:	emaste
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21466

r351963:
fusefs: coverity cleanup in the tests

Address the following defects reported by Coverity:

* Structurally dead code (CID 1404366): set m_quit before FAIL, not after

* Unchecked return value of sysctlbyname (CID 1404321)

* Unchecked return value of stat(2) (CID 1404471)

* Unchecked return value of open(2) (CID 1404402, 1404529)

* Unchecked return value of dup(2) (CID 1404478)

* Buffer overflows. These are all false positives caused by the fact that
  Coverity thinks I'm using a buffer to store strings, when in fact I'm
  really just using it to store a byte array that happens to be initialized
  with a string. I'm changing the type from char to uint8_t in the hopes
  that it will placate Coverity. (CID 1404338, 1404350, 1404367, 1404376,
  1404379, 1404381, 1404388, 1404403, 1404425, 1404433, 1404434, 1404474,
  1404480, 1404484, 1404503, 1404505)

* False positive file descriptor leak. I'm going to try to fix this with
  Coverity modeling, but I'll also change an EXPECT to ASSERT so we don't
  perform meaningless assertions after the failure. (CID 1404320, 1404324,
  1404440, 1404445).

* Unannotated file descriptor leak. This will be followed up by a Coverity
  modeling change. (CID 1404326, 1404334, 1404336, 1404357, 1404361,
  1404372, 1404391, 1404395, 1404409, 1404430, 1404448, 1404451, 1404455,
  1404457, 1404458, 1404460)

* Uninitialized variables in C++ constructors (CID 1404327, 1404346). In the
  case of m_maxphys, this actually led to part of the FUSE_INIT's response
  being set to stack garbage during the WriteCluster::clustering test.

* Uninitialized sun_len field in struct sockaddr_un (CID 1404330, 1404371,
  1404429).

Reported by:	Coverity
Reviewed by:	emaste
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21457

r352021:
fusefs: suppress some Coverity resource leak CIDs in the tests

The fusefs tests deliberately leak file descriptors.  To do otherwise would
add extra complications to the tests' mock FUSE server.  This annotation
should hopefully convince Coverity to shut up about the leaks.

Reviewed by:	uqs
Sponsored by:	The FreeBSD Foundation

r352025:
mount_fusefs: fix a segfault on memory allocation failure

Reported by:	Coverity
Coverity CID:	1354188
Sponsored by:	The FreeBSD Foundation

r352230:
fusefs: Fix iosize for FUSE_WRITE in 7.8 compat mode

When communicating with a FUSE server that implements version 7.8 (or older)
of the FUSE protocol, the FUSE_WRITE request structure is 16 bytes shorter
than normal. The protocol version check wasn't applied universally, leading
to an extra 16 bytes being sent to such servers. The extra bytes were
allocated and bzero()d, so there was no information disclosure.

Reviewed by:	emaste
MFC-With:	r350665
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21557

1 files changed, 243 insertions, 0 deletions

diff --git a/tests/sys/fs/fusefs/utils.hh b/tests/sys/fs/fusefs/utils.hh
new file mode 100644
index 000000000000..518f294fbab3
--- /dev/null
+++ b/tests/sys/fs/fusefs/utils.hh
@@ -0,0 +1,243 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
+ *
+ * Copyright (c) 2019 The FreeBSD Foundation
+ *
+ * This software was developed by BFF Storage Systems, LLC under sponsorship
+ * from the FreeBSD Foundation.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+struct _sem;
+typedef struct _sem sem_t;
+struct _dirdesc;
+typedef struct _dirdesc DIR;
+
+/* Nanoseconds to sleep, for tests that must */
+#define NAP_NS	(100'000'000)
+
+void get_unprivileged_id(uid_t *uid, gid_t *gid);
+inline void nap()
+{
+	usleep(NAP_NS / 1000);
+}
+
+bool is_unsafe_aio_enabled(void);
+
+extern const uint32_t libfuse_max_write;
+extern const uint32_t default_max_write;
+class FuseTest : public ::testing::Test {
+	protected:
+	uint32_t m_maxreadahead;
+	uint32_t m_maxwrite;
+	uint32_t m_init_flags;
+	bool m_allow_other;
+	bool m_default_permissions;
+	uint32_t m_kernel_minor_version;
+	enum poll_method m_pm;
+	bool m_push_symlinks_in;
+	bool m_ro;
+	bool m_async;
+	bool m_noclusterr;
+	bool m_nointr;
+	unsigned m_time_gran;
+	MockFS *m_mock = NULL;
+	const static uint64_t FH = 0xdeadbeef1a7ebabe;
+
+	public:
+	int m_maxbcachebuf;
+	int m_maxphys;
+
+	FuseTest():
+		m_maxreadahead(0),
+		m_maxwrite(default_max_write),
+		m_init_flags(0),
+		m_allow_other(false),
+		m_default_permissions(false),
+		m_kernel_minor_version(FUSE_KERNEL_MINOR_VERSION),
+		m_pm(BLOCKING),
+		m_push_symlinks_in(false),
+		m_ro(false),
+		m_async(false),
+		m_noclusterr(false),
+		m_nointr(false),
+		m_time_gran(1),
+		m_maxbcachebuf(0),
+		m_maxphys(0)
+	{}
+
+	virtual void SetUp();
+
+	virtual void TearDown() {
+		if (m_mock)
+			delete m_mock;
+	}
+
+	/*
+	 * Create an expectation that FUSE_ACCESS will be called once for the
+	 * given inode with the given access_mode, returning the given errno
+	 */
+	void expect_access(uint64_t ino, mode_t access_mode, int error);
+
+	/* Expect FUSE_DESTROY and shutdown the daemon */
+	void expect_destroy(int error);
+
+	/*
+	 * Create an expectation that FUSE_FLUSH will be called times times for
+	 * the given inode
+	 */
+	void expect_flush(uint64_t ino, int times, ProcessMockerT r);
+
+	/*
+	 * Create an expectation that FUSE_FORGET will be called for the given
+	 * inode.  There will be no response.  If sem is provided, it will be
+	 * posted after the operation is received by the daemon.
+	 */
+	void expect_forget(uint64_t ino, uint64_t nlookup, sem_t *sem = NULL);
+
+	/*
+	 * Create an expectation that FUSE_GETATTR will be called for the given
+	 * inode any number of times.  It will respond with a few basic
+	 * attributes, like the given size and the mode S_IFREG | 0644
+	 */
+	void expect_getattr(uint64_t ino, uint64_t size);
+
+	/*
+	 * Create an expectation that FUSE_LOOKUP will be called for the given
+	 * path exactly times times and cache validity period.  It will respond
+	 * with inode ino, mode mode, filesize size.
+	 */
+	void expect_lookup(const char *relpath, uint64_t ino, mode_t mode,
+		uint64_t size, int times, uint64_t attr_valid = UINT64_MAX,
+		uid_t uid = 0, gid_t gid = 0);
+
+	/* The protocol 7.8 version of expect_lookup */
+	void expect_lookup_7_8(const char *relpath, uint64_t ino, mode_t mode,
+		uint64_t size, int times, uint64_t attr_valid = UINT64_MAX,
+		uid_t uid = 0, gid_t gid = 0);
+
+	/*
+	 * Create an expectation that FUSE_OPEN will be called for the given
+	 * inode exactly times times.  It will return with open_flags flags and
+	 * file handle FH.
+	 */
+	void expect_open(uint64_t ino, uint32_t flags, int times);
+
+	/*
+	 * Create an expectation that FUSE_OPENDIR will be called exactly once
+	 * for inode ino.
+	 */
+	void expect_opendir(uint64_t ino);
+
+	/*
+	 * Create an expectation that FUSE_READ will be called exactly once for
+	 * the given inode, at offset offset and with size isize.  It will
+	 * return the first osize bytes from contents
+	 *
+	 * Protocol 7.8 tests can use this same expectation method because
+	 * nothing currently validates the size of the fuse_read_in struct.
+	 */
+	void expect_read(uint64_t ino, uint64_t offset, uint64_t isize,
+		uint64_t osize, const void *contents, int flags = -1);
+
+	/*
+	 * Create an expectation that FUSE_READIR will be called any number of
+	 * times on the given ino with the given offset, returning (by copy)
+	 * the provided entries
+	 */
+	void expect_readdir(uint64_t ino, uint64_t off,
+		std::vector<struct dirent> &ents);
+
+	/* 
+	 * Create an expectation that FUSE_RELEASE will be called exactly once
+	 * for the given inode and filehandle, returning success
+	 */
+	void expect_release(uint64_t ino, uint64_t fh);
+
+	/*
+	 * Create an expectation that FUSE_RELEASEDIR will be called exactly
+	 * once for the given inode
+	 */
+	void expect_releasedir(uint64_t ino, ProcessMockerT r);
+
+	/*
+	 * Create an expectation that FUSE_UNLINK will be called exactly once
+	 * for the given path, returning an errno
+	 */
+	void expect_unlink(uint64_t parent, const char *path, int error);
+
+	/*
+	 * Create an expectation that FUSE_WRITE will be called exactly once
+	 * for the given inode, at offset offset, with  size isize and buffer
+	 * contents.  Any flags present in flags_set must be set, and any
+	 * present in flags_unset must not be set.  Other flags are don't care.
+	 * It will return osize.
+	 */
+	void expect_write(uint64_t ino, uint64_t offset, uint64_t isize,
+		uint64_t osize, uint32_t flags_set, uint32_t flags_unset,
+		const void *contents);
+
+	/* Protocol 7.8 version of expect_write */
+	void expect_write_7_8(uint64_t ino, uint64_t offset, uint64_t isize,
+		uint64_t osize, const void *contents);
+
+	/*
+	 * Helper that runs code in a child process.
+	 *
+	 * First, parent_func runs in the parent process.
+	 * Then, child_func runs in the child process, dropping privileges if
+	 * desired.
+	 * Finally, fusetest_fork returns.
+	 *
+	 * # Returns
+	 *
+	 * fusetest_fork may SKIP the test, which the caller should detect with
+	 * the IsSkipped() method.  If not, then the child's exit status will
+	 * be returned in status.
+	 */
+	void fork(bool drop_privs, int *status,
+		std::function<void()> parent_func,
+		std::function<int()> child_func);
+
+	/*
+	 * Deliberately leak a file descriptor.
+	 *
+	 * Closing a file descriptor on fusefs would cause the server to
+	 * receive FUSE_CLOSE and possibly FUSE_INACTIVE.  Handling those
+	 * operations would needlessly complicate most tests.  So most tests
+	 * deliberately leak the file descriptors instead.  This method serves
+	 * to document the leakage, and provide a single point of suppression
+	 * for static analyzers.
+	 */
+	/* coverity[+close: arg-0] */
+	static void leak(int fd __unused) {}
+
+	/*
+	 * Deliberately leak a DIR* pointer
+	 *
+	 * See comments for FuseTest::leak
+	 */
+	static void leakdir(DIR* dirp __unused) {}
+};