Build system

  • Uses pipmake, a tiny build system which is pip-compatible, but forwards pip commands to a Makefile (e.g. pip install forwards to make wheel).

  • The Makefile runs the script makefile_helper.py, which contains miscellaneous logic that’s more convenient to write in python than in Makefile language. The output of this script is a file makefile_helper.out, containing variable declarations in Makefile language.

  • Build the whole project with make -j 32. Compile time is high, so make src_lib/FILE.o may be useful when working on src_lib/FILE.cu.

  • If you add a new source file, see comments near the top of Makefile for instructions on how to modify the makefile.

Autogenerated kernels

  • Source files of the form src_lib/autogenerated_kernels/*.cu are autogenerated during the build process, with python autogenerate_kernel.py src_lib/autogenerated_kernels/FILE.cu.

  • The script makefile_helper.py decides which source files to autogenerate, and writes a list of filenames to makefile_helper.out.

  • Individual source files can be generated or compiled with make src_lib/autogenerated_kernels/filename.{cu,o}.

  • Each source file “registers” its precompiled kernel at library init time, and the registered kernels are found at runtime. (See examples in PeakFindingKernel.hpp.)

gRPC

  • Proto files live in grpc/ (currently just grpc/frb_search.proto).

  • make grpc generates both C++ and Python files from .proto files.

  • C++ generated files (*.pb.{h,cc}, *.grpc.pb.{h,cc}) are output to grpc/.

  • Python generated files (*_pb2.py, *_pb2_grpc.py) are output to pirate_frb/rpc/grpc/.

  • C++ generation uses protoc with grpc_cpp_plugin. Python generation uses grpc_tools.protoc followed by protol (protoletariat) to fix relative imports.

  • Sentinel files (*.protoc_cpp_sentinel, *.protoc_python_sentinel) ensure each protoc invocation runs exactly once despite producing multiple output files.

  • The shared library links against -lgrpc++ and -lprotobuf.

File Structure

include/pirate/*.hpp                - Public C++ headers
src_lib/*.cpp                       - Pure C++ implementations
src_lib/*.cu                        - CUDA implementations (kernels)
src_lib/autogenerated_kernels/*.cu  - Generated kernels (DO NOT EDIT)
pirate_frb/cuda_generator/*.py      - Python code that generates CUDA kernels
pirate_frb/*.py                     - High-level python interface
pirate_frb/rpc/                     - RPC client code (FrbClient.py)
pirate_frb/rpc/grpc/                - Generated Python protobuf/gRPC stubs
grpc/*.proto                        - Protocol buffer definitions
grpc/*.pb.{h,cc}                    - Generated C++ protobuf/gRPC code
bin/                                - Compiled executables
lib/libpirate.so                    - Shared library