diff --git a/docs/contributor/CONTRIBUTING.md b/docs/contributor/CONTRIBUTING.md index d98a03a95b..0a67f37573 100644 --- a/docs/contributor/CONTRIBUTING.md +++ b/docs/contributor/CONTRIBUTING.md @@ -17,42 +17,192 @@ For full editing support, run `mx vscodeinit` and open the `graalpython.code-wor ### Setting up on your machine -The first thing you want to do is to set up [`mx`](https://github.com/graalvm/mx.git). -This is the build tool we use to develop GraalVM languages. +#### General prerequisites + +All platforms require: + +* Git +* CPython 3.8 or newer, which is required to run `mx` +* [Maven](https://maven.apache.org/), with its `bin` directory on your `PATH` +* [`mx`](https://github.com/graalvm/mx.git), the build tool used to develop GraalVM languages +* A suitable JDK +* CMake 3.22 or newer + +`mx` provides Ninja when it is not already available, so Ninja does not need to +be installed separately. + +After installing the platform-specific prerequisites described below, clone `mx`: + ```shell git clone https://github.com/graalvm/mx.git ``` -Make sure to add the `mx` directory to your `PATH`. -Use `mx` to get additional projects at the right versions. -From within your `graalpython` checkout, run: +Add the `mx` directory to your `PATH`. + +Use `mx` to get additional projects at the right versions. From within your +`graalpython` checkout, run: + ``` mx sforceimport ``` You can then download a suitable JDK: -```bash + +```shell mx -p ../graal/vm --env ce-python fetch-jdk -A --jdk-id labsjdk-ce-latest ``` -Make sure that the `JAVA_HOME` environment variable is set: +Set the `JAVA_HOME` environment variable as described for your platform below. + +#### Linux prerequisites + +For building GraalPy, you will also need some native build tools and libraries. +On a Debian-based system, install: + +```bash +sudo apt install build-essential libc++-12-dev zlib1g-dev cmake +``` + +The `python-libzsupport` project uses the zlib headers and library from +`zlib1g-dev`. `python-libposix` uses the libc development headers and links +against `libutil` and `librt`; these are provided by `libc6-dev`, which is +installed through `build-essential`. +The bzip2 and XZ sources used by `python-libbz2` and `python-liblzma` are +downloaded by `mx`, so `libbz2-dev` and `liblzma-dev` are not required. +If your distribution provides a CMake version older than 3.22, install a newer +version separately. + +Set `JAVA_HOME`: + ```bash -export JAVA_HOME="${HOME}/.mx/jdks/labsjdk-ce-latest +export JAVA_HOME="${HOME}/.mx/jdks/labsjdk-ce-latest" ``` -(Or on Windows) +#### Windows prerequisites + +GraalPy can be built natively on 64-bit Windows. Use +[Git for Windows](https://gitforwindows.org/) for the general Git prerequisite. +In addition, install: + +* Visual Studio 2022 with the **Desktop development with C++** workload, including + the MSVC x64 build tools and a Windows SDK + +The Windows SDK provides the headers and `Ws2_32.lib` required by +`python-libposix`. CMake 3.22.2 is tested. Newer Visual Studio and CMake +versions may work, but are not tested. + +GraalPy's checkout and generated build output require symbolic links and long paths. +First, enable **Developer Mode** in Windows Settings under **System > For developers**. +Developer Mode allows an unelevated process to create symbolic links. If Developer +Mode is controlled by organizational policy, builds must instead run from an +elevated shell. + +Next, enable Win32 long paths from an Administrator PowerShell: + +```powershell +Set-ItemProperty ` + -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" ` + -Name LongPathsEnabled ` + -Type DWord ` + -Value 1 ``` -$env:JAVA_HOME="$HOME\.mx\jdks\labsjdk-ce-latest" + +Verify that the command prints `1`: + +```powershell +Get-ItemPropertyValue ` + -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" ` + -Name LongPathsEnabled ``` -For building GraalPy, you will also need some native build tools and libraries. On a Debian based system, install: -```bash -sudo apt install build-essential libc++-12-dev zlib1g-dev cmake +Close and reopen applications after changing this setting; a reboot may be needed +because applications can cache it. Configure Git to support both long paths and +symbolic links: + +```powershell +git config --global core.longpaths true +git config --global core.symlinks true +``` + +For a new checkout, explicitly enable symbolic links while cloning: + +```powershell +git clone -c core.symlinks=true https://github.com/oracle/graalpython.git +``` + +Git for Windows may have written `core.symlinks=false` into an existing checkout's +local `.git/config`, which overrides the global setting. Check all effective values +from inside the checkout: + +```powershell +git config --show-origin --show-scope --get-all core.symlinks +``` + +If the local value is `false`, repair the configuration: + +```powershell +git config --local core.symlinks true +``` + +Files stored as symbolic links have mode `120000` in Git: + +```powershell +git ls-files -s | Select-String '^120000\s' +``` + +For example, verify the `common.json` link: + +```powershell +Get-Item -Force .\common.json | Format-List LinkType,Target +``` + +It should report `LinkType: SymbolicLink`. If it is a regular file containing only +`./ci/graal/common.json`, changing the configuration does not automatically +re-materialize it. Replace it after enabling Developer Mode and `core.symlinks`: + +```powershell +Remove-Item -LiteralPath .\common.json +git restore --source=HEAD --worktree -- .\common.json +``` + +Run the build in a 64-bit Visual Studio Developer PowerShell. A Developer PowerShell +can be initialized from an ordinary PowerShell without relying on a +version-specific Visual Studio installation path: + +```powershell +$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" +$vsPath = & $vswhere -latest -products * ` + -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 ` + -property installationPath +& "$vsPath\Common7\Tools\Launch-VsDevShell.ps1" -Arch amd64 -HostArch amd64 +``` + +Verify the target architecture and tools before building: + +```powershell +$env:VSCMD_ARG_TGT_ARCH +cl +where.exe cl +where.exe dumpbin +cmake --version +python --version +$env:INCLUDE +$env:LIB +``` + +`VSCMD_ARG_TGT_ARCH` and the `cl` banner must indicate `x64`, not `x86`. +Both `INCLUDE` and `LIB` must contain the Visual C++ and Windows SDK paths. + +Set `JAVA_HOME`: + +```powershell +$env:JAVA_HOME="$HOME\.mx\jdks\labsjdk-ce-latest" ``` -(On Windows, make sure you are running in a Visual Studio Developer Powershell, that should have everything you need.) +Make sure you are running in the x64 Visual Studio Developer PowerShell configured +above when building. -Lastly, download maven, extract it and include it on your `PATH`. +#### Building GraalPy Once you have all the necessary tools, you can run `mx python-jvm` in this repository. This will initially download the required dependencies next to the repository and build Python.