aboutsummaryrefslogtreecommitdiff
path: root/tools
diff options
context:
space:
mode:
authorcrueter <crueter@eden-emu.dev>2025-10-20 03:43:15 +0200
committercrueter <crueter@eden-emu.dev>2025-10-20 03:43:15 +0200
commit2e9dbe3f1d8133570cd9db92aa05769733451029 (patch)
tree51cb00c2f2a89e844b824c6403faa3ad38ea5c76 /tools
parentd7cd7c6313e7fc5f1ccf73e48228c97b3246d99a (diff)
[cmake, tools, docs] Update CPMUtil (#2790)
updates CPMUtil to [`8f9b39fc98`](https://git.crueter.xyz/CMake/CPMUtil/commit/8f9b39fc980fc7079c7c4471c19f219180abe368) Makes the tooling/docs better and enforces usage of tar.gz everywhere as opposed to zips Signed-off-by: crueter <crueter@eden-emu.dev> Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/2790 Reviewed-by: CamilleLaVey <camillelavey99@gmail.com> Reviewed-by: MaranBr <maranbr@eden-emu.dev>
Diffstat (limited to 'tools')
-rwxr-xr-xtools/cpm/README.md24
-rwxr-xr-xtools/cpm/check-hash-all.sh4
-rwxr-xr-xtools/cpm/check-hash.sh53
-rwxr-xr-xtools/cpm/check-updates-all.sh4
-rwxr-xr-xtools/cpm/check-updates.sh45
-rwxr-xr-xtools/cpm/common.sh15
-rwxr-xr-xtools/cpm/download.sh3
-rwxr-xr-xtools/cpm/fetch-all.sh2
-rwxr-xr-xtools/cpm/fetch.sh30
-rwxr-xr-xtools/cpm/format.sh5
-rwxr-xr-xtools/cpm/hash.sh94
-rwxr-xr-xtools/cpm/package.sh9
-rwxr-xr-xtools/cpm/which.sh4
-rwxr-xr-xtools/shellcheck.sh10
-rwxr-xr-xtools/update-cpm.sh14
15 files changed, 240 insertions, 76 deletions
diff --git a/tools/cpm/README.md b/tools/cpm/README.md
index acd7444518..a28c5d37cf 100755
--- a/tools/cpm/README.md
+++ b/tools/cpm/README.md
@@ -2,7 +2,13 @@
These are supplemental shell scripts for CPMUtil aiming to ease maintenance burden for sanity checking, updates, prefetching, formatting, and standard operations done by these shell scripts, all in one common place.
-All scripts are POSIX-compliant.
+All scripts are POSIX-compliant. If something doesn't work on your shell, ensure it's POSIX-compliant.
+* If your shell doesn't support `$(...)` syntax, you've got bigger problems to worry about.
+<!-- TOC -->
+- [Meta](#meta)
+- [Simple Utilities](#simple-utilities)
+- [Functional Utilities](#functional-utilities)
+<!-- /TOC -->
## Meta
@@ -39,9 +45,8 @@ These scripts don't really have any functionality, they just help you out a bit
* Inputs:
- The repository (e.g. fmtlib/fmt)
- The sha or tag (e.g. v1.0.1)
- - `GIT_HOST`: What git host to use (default github.com)
- - `USE_TAG`: Set to "true" if the second argument is a tag instead of a sha
- - `ARTIFACT`: The artifact to download, if using a tag. Set to null or empty to use the tag source archive instead
+ - `-g <GIT_HOST>` or `--host <GIT_HOST>`: What git host to use (default github.com)
+ - `-a <ARTIFACT>` or `--artifact <ARTIFACT>`: The artifact to download. Set to null or empty to use a source archive instead
* Output: the SHA512 sum of the package
- `url-hash.sh`: Determine the hash of a URL
* Input: the URL
@@ -51,7 +56,7 @@ These scripts don't really have any functionality, they just help you out a bit
These modify the CPM cache or cpmfiles. Each allows you to input all the packages to act on, as well as a `<scriptname>-all.sh` that acts upon all available packages.
-For the update and hash scripts, set `UPDATE=true` to update the cpmfile with the new version or hash. Beware: if the hash is `cf83e1357...` that means you got a 404 error!
+Beware: if a hash is `cf83e1357...` that means you got a 404 error!
- `fetch.sh`: Prefetch a package according to its cpmfile definition
* Packages are fetched to the `.cache/cpm` directory by default, following the CPMUtil default.
@@ -61,11 +66,14 @@ For the update and hash scripts, set `UPDATE=true` to update the cpmfile with th
- `check-updates.sh`: Check a package for available updates
* This only applies to packages that utilize tags.
* If the tag is a format string, the `git_version` is acted upon instead.
- * Setting `FORCE=true` will forcefully update every package and its hash, even if they are on the latest version (`UPDATE` must also be true)
+ * Specifying `-f` or `--force` will forcefully update the package and its hash, even if it's on on the latest version.
+ * Alternatively, only specify `-u` or `--update` to update packages that have new versions available.
* This script generally runs fast.
- * Packages that should skip updates (e.g. older versions or packages with poorly-made tag structures... looking at you mbedtls) may specify `"skip_updates": true` in their cpmfile definition. This is unnecessary for untagged (e.g. sha or bare URL) packages.
+ * Packages that should skip updates (e.g. older versions, OR packages with poorly-made tag structures... looking at you mbedtls) may specify `"skip_updates": true` in their cpmfile definition. This is unnecessary for untagged (e.g. sha or bare URL) packages.
- `check-hashes.sh`: Check a package's hash
+ * Specifying `-f` or `--force` will update the package's hash even if it's not mismatched.
+ * Alternatively, specify `-u` or `--update` to only fix mismatched hashes.
* This only applies to packages with hardcoded hashes, NOT ones that use hash URLs.
- * This script will take a looooooooooooooong time. This is operationally equivalent to a prefetch, and thus checking all hashes will take a while--but it's worth it! Just make sure you're not using dial-up.
+ * This script will take a long time. This is operationally equivalent to a prefetch, and thus checking all hashes will take a while--but it's worth it! Just make sure you're not using dial-up.
You are recommended to run sanity hash checking for every pull request and commit, and weekly update checks. \ No newline at end of file
diff --git a/tools/cpm/check-hash-all.sh b/tools/cpm/check-hash-all.sh
index fd8c270392..6236ba8bf2 100755
--- a/tools/cpm/check-hash-all.sh
+++ b/tools/cpm/check-hash-all.sh
@@ -1,4 +1,4 @@
-#!/bin/bash -e
+#!/bin/sh -e
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
@@ -7,4 +7,4 @@
. tools/cpm/common.sh
# shellcheck disable=SC2086
-tools/cpm/check-hash.sh $LIBS \ No newline at end of file
+tools/cpm/check-hash.sh "$@" $LIBS \ No newline at end of file
diff --git a/tools/cpm/check-hash.sh b/tools/cpm/check-hash.sh
index 85c60aad8c..868d382060 100755
--- a/tools/cpm/check-hash.sh
+++ b/tools/cpm/check-hash.sh
@@ -3,18 +3,49 @@
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-# env vars:
-# - UPDATE: fix hashes if needed
-
# shellcheck disable=SC1091
. tools/cpm/common.sh
RETURN=0
-for PACKAGE in "$@"
-do
+usage() {
+ cat << EOF
+Usage: $0 [uf] [PACKAGE]...
+Check the hash of a specific package or packages.
+If a hash mismatch occurs, this script will print the corrected hash of the package.
+
+Options:
+ -u, --update Correct the package's hash if it's a mismatch
+
+ -f, --force Update the package's hash anyways (implies -u)
+
+Note that this procedure will usually take a long time
+depending on the number and size of dependencies.
+
+This project has defined the following as valid cpmfiles:
+EOF
+
+ for file in $CPMFILES; do
+ echo "- $file"
+ done
+
+ exit $RETURN
+}
+
+while true; do
+ case "$1" in
+ (-uf|-f|--force) UPDATE=true; FORCE=true; shift; continue ;;
+ (-u|--update) UPDATE=true; shift; continue ;;
+ (-h) usage ;;
+ ("$0") break ;;
+ ("") break ;;
+ esac
+
+ PACKAGE="$1"
+
+ shift
+
export PACKAGE
- # shellcheck disable=SC1091
. tools/cpm/package.sh
if [ "$CI" != null ]; then
@@ -31,11 +62,13 @@ do
export USE_TAG=true
ACTUAL=$(tools/cpm/url-hash.sh "$DOWNLOAD")
- # shellcheck disable=SC2028
- [ "$ACTUAL" != "$HASH" ] && echo "-- * Expected $HASH" && echo "-- * Got $ACTUAL" && [ "$UPDATE" != "true" ] && RETURN=1
+ if [ "$ACTUAL" != "$HASH" ]; then
+ echo "-- * Expected $HASH"
+ echo "-- * Got $ACTUAL"
+ [ "$UPDATE" != "true" ] && RETURN=1
+ fi
- if [ "$UPDATE" = "true" ] && [ "$ACTUAL" != "$HASH" ]; then
- # shellcheck disable=SC2034
+ if { [ "$UPDATE" = "true" ] && [ "$ACTUAL" != "$HASH" ]; } || [ "$FORCE" = "true" ]; then
NEW_JSON=$(echo "$JSON" | jq ".hash = \"$ACTUAL\"")
export NEW_JSON
diff --git a/tools/cpm/check-updates-all.sh b/tools/cpm/check-updates-all.sh
index a6eda58bac..473c067ff6 100755
--- a/tools/cpm/check-updates-all.sh
+++ b/tools/cpm/check-updates-all.sh
@@ -1,4 +1,4 @@
-#!/bin/bash -e
+#!/bin/sh -e
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
@@ -7,4 +7,4 @@
. tools/cpm/common.sh
# shellcheck disable=SC2086
-tools/cpm/check-updates.sh $LIBS \ No newline at end of file
+tools/cpm/check-updates.sh "$@" $LIBS \ No newline at end of file
diff --git a/tools/cpm/check-updates.sh b/tools/cpm/check-updates.sh
index bdccf96ca2..7fa1904e4e 100755
--- a/tools/cpm/check-updates.sh
+++ b/tools/cpm/check-updates.sh
@@ -3,21 +3,51 @@
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-# env vars:
-# - UPDATE: update if available
-# - FORCE: forcefully update
-
# shellcheck disable=SC1091
. tools/cpm/common.sh
RETURN=0
filter() {
- TAGS=$(echo "$TAGS" | jq "[.[] | select(.name | test(\"$1\"; \"i\") | not)]") # vulkan
+ TAGS=$(echo "$TAGS" | jq "[.[] | select(.name | test(\"$1\"; \"i\") | not)]")
}
-for PACKAGE in "$@"
-do
+usage() {
+ cat << EOF
+Usage: $0 [uf] [PACKAGE]...
+Check a specific package or packages for updates.
+
+Options:
+ -u, --update Update the package if a new version is available.
+ This will also update the hash if provided.
+
+ -f, --force Forcefully update the package version (implies -u)
+ This is seldom useful, and should only be used in cases of
+ severe corruption.
+
+This project has defined the following as valid cpmfiles:
+EOF
+
+ for file in $CPMFILES; do
+ echo "- $file"
+ done
+
+ exit $RETURN
+}
+
+while true; do
+ case "$1" in
+ (-uf|--force) UPDATE=true; FORCE=true; continue ;;
+ (-u|--update) UPDATE=true; continue ;;
+ (-h) usage ;;
+ ("$0") break ;;
+ ("") break ;;
+ esac
+
+ PACKAGE="$1"
+
+ shift
+
export PACKAGE
# shellcheck disable=SC1091
. tools/cpm/package.sh
@@ -67,7 +97,6 @@ do
echo "-- * Version $LATEST available, current is $TAG"
- export USE_TAG=true
HASH=$(tools/cpm/hash.sh "$REPO" "$LATEST")
echo "-- * New hash: $HASH"
diff --git a/tools/cpm/common.sh b/tools/cpm/common.sh
index 4aff058bdc..92a88f0c9b 100755
--- a/tools/cpm/common.sh
+++ b/tools/cpm/common.sh
@@ -7,22 +7,21 @@
# CHANGE THESE FOR YOUR PROJECT! #
##################################
-# Which directories to search
-DIRS=". src"
-
# How many levels to go (3 is 2 subdirs max)
MAXDEPTH=3
-# shellcheck disable=SC2038
+# For your project you'll want to change this to define what dirs you have cpmfiles in
+# Remember to account for the MAXDEPTH variable!
+# Adding ./ before each will help to remove duplicates
+[ -z "$CPMFILES" ] && CPMFILES=$(find . ./src -maxdepth "$MAXDEPTH" -name cpmfile.json | sort | uniq)
+
# shellcheck disable=SC2016
-# shellcheck disable=SC2086
-[ -z "$PACKAGES" ] && PACKAGES=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json | xargs jq -s 'reduce .[] as $item ({}; . * $item)')
+[ -z "$PACKAGES" ] && PACKAGES=$(echo "$CPMFILES" | xargs jq -s 'reduce .[] as $item ({}; . * $item)')
-# For your project you'll want to change the PACKAGES call to include whatever locations you may use (externals, src, etc.)
-# Always include .
LIBS=$(echo "$PACKAGES" | jq -j 'keys_unsorted | join(" ")')
export PACKAGES
+export CPMFILES
export LIBS
export DIRS
export MAXDEPTH
diff --git a/tools/cpm/download.sh b/tools/cpm/download.sh
index 426f1f51e6..315f01d54b 100755
--- a/tools/cpm/download.sh
+++ b/tools/cpm/download.sh
@@ -3,9 +3,6 @@
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-# env vars:
-# - UPDATE: fix hashes if needed
-
# shellcheck disable=SC1091
. tools/cpm/common.sh
diff --git a/tools/cpm/fetch-all.sh b/tools/cpm/fetch-all.sh
index 5c41b5d080..3f5b605860 100755
--- a/tools/cpm/fetch-all.sh
+++ b/tools/cpm/fetch-all.sh
@@ -7,4 +7,4 @@
. tools/cpm/common.sh
# shellcheck disable=SC2086
-tools/cpm/fetch.sh $LIBS \ No newline at end of file
+tools/cpm/fetch.sh "$@" $LIBS \ No newline at end of file
diff --git a/tools/cpm/fetch.sh b/tools/cpm/fetch.sh
index bf45676cfa..684fc1107a 100755
--- a/tools/cpm/fetch.sh
+++ b/tools/cpm/fetch.sh
@@ -18,9 +18,33 @@ ROOTDIR="$PWD"
TMP=$(mktemp -d)
-# shellcheck disable=SC2034
-for PACKAGE in "$@"
-do
+usage() {
+ cat << EOF
+Usage: $0 [PACKAGE]...
+Fetch the specified package or packages from their defined download locations.
+If the package is already cached, it will not be re-fetched.
+
+This project has defined the following as valid cpmfiles:
+EOF
+
+ for file in $CPMFILES; do
+ echo "- $file"
+ done
+
+ exit 0
+}
+
+while true; do
+ case "$1" in
+ (-h) usage ;;
+ ("$0") break ;;
+ ("") break ;;
+ esac
+
+ PACKAGE="$1"
+
+ shift
+
export PACKAGE
# shellcheck disable=SC1091
. tools/cpm/package.sh
diff --git a/tools/cpm/format.sh b/tools/cpm/format.sh
index 8d99b4796b..e61773d4a8 100755
--- a/tools/cpm/format.sh
+++ b/tools/cpm/format.sh
@@ -6,10 +6,7 @@
# shellcheck disable=SC1091
. tools/cpm/common.sh
-# shellcheck disable=SC2086
-FILES=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json)
-
-for file in $FILES; do
+for file in $CPMFILES; do
jq --indent 4 < "$file" > "$file".new
mv "$file".new "$file"
done
diff --git a/tools/cpm/hash.sh b/tools/cpm/hash.sh
index 27061bd9a4..76c5bdd5a8 100755
--- a/tools/cpm/hash.sh
+++ b/tools/cpm/hash.sh
@@ -6,20 +6,94 @@
# usage: hash.sh repo tag-or-sha
# env vars: GIT_HOST, USE_TAG (use tag instead of sha), ARTIFACT (download artifact with that name instead of src archive)
-REPO="$1"
-[ -z "$GIT_HOST" ] && GIT_HOST=github.com
+RETURN=0
+
+usage() {
+ cat <<EOF
+Usage: $0 [-a|--artifact ARTIFACT] [-g|--host GIT_HOST] [REPO] [REF]
+Get the hash of a package.
+
+REPO must be in the form of OWNER/REPO, and REF must be a commit sha, branch, or tag.
+
+Options:
+ -g, --host <GIT_HOST> What Git host to use (defaults to github.com)
+
+ -a, --artifact <ARTIFACT> The artifact to download (implies -t)
+ If -t is specified but not -a, fetches a tag archive.
+ If ARTIFACT is specified but is null,
+
+EOF
+ exit "$RETURN"
+}
+
+die() {
+ echo "$@" >&2
+ RETURN=1 usage
+}
+
+artifact() {
+ if [ $# -lt 2 ]; then
+ die "You must specify a valid artifact."
+ fi
+
+ shift
+
+ ARTIFACT="$1"
+}
+
+host() {
+ if [ $# -lt 2 ]; then
+ die "You must specify a valid Git host."
+ fi
+
+ shift
+
+ GIT_HOST="$1"
+}
+
+# this is a semi-hacky way to handle long/shortforms
+while true; do
+ case "$1" in
+ -[a-z]*)
+ opt=$(echo "$1" | sed 's/^-//')
+ while [ -n "$opt" ]; do
+ # cut out first char from the optstring
+ char=$(echo "$opt" | cut -c1)
+ opt=$(echo "$opt" | cut -c2-)
+
+ case "$char" in
+ a) artifact "$@" ;;
+ g) host "$@" ;;
+ h) usage ;;
+ *) die "Invalid option -$char" ;;
+ esac
+ done
+
+ ;;
+ --artifact) artifact "$@" ;;
+ --host) host "$@" ;;
+ --help) usage ;;
+ --*) die "Invalid option $1" ;;
+ "$0" | "") break ;;
+ *)
+ { [ -z "$REPO" ] && REPO="$1"; } || REF="$1"
+ ;;
+ esac
+
+ shift
+done
+
+[ -z "$REPO" ] && die "A valid repository must be provided."
+[ -z "$REF" ] && die "A valid reference must be provided."
+
+GIT_HOST=${GIT_HOST:-github.com}
GIT_URL="https://$GIT_HOST/$REPO"
-if [ "$USE_TAG" = "true" ]; then
- if [ -z "$ARTIFACT" ] || [ "$ARTIFACT" = "null" ]; then
- URL="${GIT_URL}/archive/refs/tags/$2.tar.gz"
- else
- URL="${GIT_URL}/releases/download/$2/${ARTIFACT}"
- fi
+if [ -z "$ARTIFACT" ] || [ "$ARTIFACT" = "null" ]; then
+ URL="${GIT_URL}/archive/$REF.tar.gz"
else
- URL="${GIT_URL}/archive/$2.zip"
+ URL="${GIT_URL}/releases/download/$REF/$ARTIFACT"
fi
SUM=$(wget -q "$URL" -O - | sha512sum)
-
echo "$SUM" | cut -d " " -f1
diff --git a/tools/cpm/package.sh b/tools/cpm/package.sh
index d82b2fcbe9..79c71c18a7 100755
--- a/tools/cpm/package.sh
+++ b/tools/cpm/package.sh
@@ -3,9 +3,6 @@
# SPDX-FileCopyrightText: 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-# env vars:
-# - UPDATE: fix hashes if needed
-
# shellcheck disable=SC1091
. tools/cpm/common.sh
@@ -14,7 +11,7 @@
# shellcheck disable=SC2153
JSON=$(echo "$PACKAGES" | jq -r ".\"$PACKAGE\" | select( . != null )")
-[ -z "$JSON" ] && echo "!! No cpmfile definition for $PACKAGE" && exit 1
+[ -z "$JSON" ] && echo "!! No cpmfile definition for $PACKAGE" >&2 && exit 1
# unset stuff
export PACKAGE_NAME="null"
@@ -134,13 +131,13 @@ elif [ "$REPO" != "null" ]; then
DOWNLOAD="${GIT_URL}/archive/refs/tags/${TAG}.tar.gz"
fi
elif [ "$SHA" != "null" ]; then
- DOWNLOAD="${GIT_URL}/archive/${SHA}.zip"
+ DOWNLOAD="${GIT_URL}/archive/${SHA}.tar.gz"
else
if [ "$BRANCH" = null ]; then
BRANCH=master
fi
- DOWNLOAD="${GIT_URL}/archive/refs/heads/${BRANCH}.zip"
+ DOWNLOAD="${GIT_URL}/archive/refs/heads/${BRANCH}.tar.gz"
fi
else
echo "!! No repo or URL defined for $PACKAGE"
diff --git a/tools/cpm/which.sh b/tools/cpm/which.sh
index c936d0a97f..11d9b84156 100755
--- a/tools/cpm/which.sh
+++ b/tools/cpm/which.sh
@@ -8,8 +8,8 @@
. tools/cpm/common.sh
# shellcheck disable=SC2086
-JSON=$(find $DIRS -maxdepth "$MAXDEPTH" -name cpmfile.json -exec grep -l "$1" {} \;)
+JSON=$(echo "$CPMFILES" | xargs grep -l "$1")
-[ -z "$JSON" ] && echo "!! No cpmfile definition for $1"
+[ -z "$JSON" ] && echo "!! No cpmfile definition for $1" >&2 && exit 1
echo "$JSON" \ No newline at end of file
diff --git a/tools/shellcheck.sh b/tools/shellcheck.sh
index 719c717cf2..e5c1d63dfb 100755
--- a/tools/shellcheck.sh
+++ b/tools/shellcheck.sh
@@ -1,11 +1,7 @@
#!/bin/sh -e
-# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
+# SPDX-FileCopyrightText: Copyright 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-# fd is slightly faster on NVMe (the syntax sux though)
-if command -v fd > /dev/null; then
- fd . tools -esh -x shellcheck
-else
- find tools -name "*.sh" -exec shellcheck -s sh {} \;
-fi \ No newline at end of file
+# do NOT use fd in scripts, PLEASE
+find tools -name "*.sh" -exec shellcheck -s sh {} \;
diff --git a/tools/update-cpm.sh b/tools/update-cpm.sh
index 8bd8df2b83..6ca23fc2d6 100755
--- a/tools/update-cpm.sh
+++ b/tools/update-cpm.sh
@@ -1,6 +1,16 @@
#!/bin/sh -e
-# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
+# SPDX-FileCopyrightText: Copyright 2025 crueter
# SPDX-License-Identifier: GPL-3.0-or-later
-wget -O CMakeModules/CPM.cmake https://github.com/cpm-cmake/CPM.cmake/releases/latest/download/CPM.cmake
+# updates CPMUtil, its docs, and related tools from the latest release
+
+if command -v zstd > /dev/null; then
+ EXT=tar.zst
+else
+ EXT=tar.gz
+fi
+
+wget "https://git.crueter.xyz/CMake/CPMUtil/releases/download/continuous/CPMUtil.$EXT"
+tar xf "CPMUtil.$EXT"
+rm "CPMUtil.$EXT"