Build system

  • Uses pipmake, a tiny build system which is pip-compatible, but forwards pip commands to a Makefile (e.g. pip install forwards to make build_wheel).

  • The Makefile runs the script makefile_helper.py, which contains miscellaneous logic that’s more convenient to write in python than in Makefile language. The output of this script is a file makefile_helper.out, containing variable declarations in Makefile language.

  • Build the whole project with make -j 32. Compile time is high, so make src_lib/FILE.o may be useful when working on src_lib/FILE.cu.

  • If you add a new source file, see comments near the top of Makefile for instructions on how to modify the makefile.

  • The Makefile should work automatically with conda. If you’re not using conda, you may need to override some makefile vars (see below).

Autogenerated kernels

  • Source files of the form src_lib/autogenerated_kernels/*.cu are autogenerated during the build process, with python autogenerate_kernel.py src_lib/autogenerated_kernels/FILE.cu.

  • The script makefile_helper.py decides which source files to autogenerate, and writes a list of filenames to makefile_helper.out.

  • Individual source files can be generated or compiled with make src_lib/autogenerated_kernels/filename.{cu,o}.

  • Each source file “registers” its precompiled kernel at library init time, and the registered kernels are found at runtime. (See examples in PeakFindingKernel.hpp.)

gRPC

  • Proto files live in grpc/: frb_search.proto, frb_grouper.proto, frb_sifter.proto.

  • make grpc generates both C++ and Python files from .proto files.

  • C++ generated files (*.pb.{h,cc}, *.grpc.pb.{h,cc}) are output to grpc/.

  • Python generated files (*_pb2.py, *_pb2_grpc.py) are output to pirate_frb/rpc/grpc/.

  • C++ generation uses protoc with grpc_cpp_plugin. Python generation uses grpc_tools.protoc followed by protol (protoletariat) to fix relative imports.

  • Each protoc invocation emits several files at once (the four C++ outputs, or both Python stubs), so the rules are grouped multi-target pattern rules: one rule whose targets are all of those outputs, run once per .proto.

  • The shared library links against -lgrpc++ and -lprotobuf.

File Structure

include/pirate/*.hpp                - Public C++ headers
src_lib/*.cpp                       - Pure C++ implementations
src_lib/*.cu                        - CUDA implementations (kernels)
src_lib/autogenerated_kernels/*.cu  - Generated kernels (DO NOT EDIT)
pirate_frb/cuda_generator/*.py      - Python code that generates CUDA kernels
pirate_frb/*.py                     - High-level python interface
pirate_frb/rpc/                     - RPC client code (FrbSearchClient.py)
pirate_frb/rpc/grpc/                - Generated Python protobuf/gRPC stubs
grpc/                               - gRPC .proto sources and generated C++ stubs (*.pb.{h,cc})
configs/                            - YAML config files (frb_server, xengine, hwtest, dedispersion)
notes/                              - Developer-facing markdown notes
lib/libpirate.so                    - Shared library

Overriding Makefile variables

The Makefile’s compiler, flags, and include paths can be overridden without editing it, either from an environment variable or from a gitignored config.mk in the repo root. Conda users normally need none of these: cufftdx (mathdx), yaml-cpp, asdf, etc. are found automatically under $CONDA_PREFIX. A non-conda build typically just sets EXTRA_INCDIRS.

  • NVCC_OPT: e.g. -O0 -g. nvcc optimization flags (default -O3); use for a debug build.

  • EXTRA_INCDIRS: e.g. -I/usr/local/include -I/home/dstn/nvidia/mathdx/26.03/include. Extra -I dirs for hand-installed deps (mathdx, system asdf) on non-conda machines.

  • NVCC: e.g. /usr/local/bin/nvcc. The nvcc program, e.g. to point at a different toolkit; its flags live in NVCCFLAGS.

  • NVCCFLAGS: e.g. -std=c++20 .... All of nvcc’s flags; usually tweak NVCC_OPT instead.

  • NVCC_ARCH: e.g. -gencode arch=compute_90,code=sm_90. GPU-architecture flags (default targets sm_80/86/89).

  • NVCC_DEPFLAGS: e.g. -MMD -MP. Dependency-file generation flags (the default).

  • PYTHON: e.g. python3.11. Python interpreter used by the build (default python3).

Precedence, highest first: make VAR=... on the command line, then config.mk, then the environment, then the Makefile defaults.

For a persistent non-conda setup, put the overrides in config.mk (gitignored). For example, Dustin’s config.mk is:

EXTRA_INCDIRS = -I/usr/local/include -I/home/dstn/nvidia/mathdx/26.03/include
# NVCC_OPT = -O0 -g