freeipa

#6418 Build system refactoring
Closed Opened by pspacek.

Closed
Reopen Issue

The build system in FreeIPA 4.4 has following problems:

  • It is ineffective. This costs developer time every time we build IPA. Every developer typically builds IPA several times a day. Also, it slows down CI because it needs to build the packages as well.
  • The build system does not directly support rapid development approaches: Every patch-copy-test cycle requires either manual work (which is error prone) or full build (which is very slow). It slows down rapid prototyping.
    • Repeated build with minimal changes (like changing one file) takes long time, which prevents us from doing rapid devel cycle.
    • Theoretically the process should be very fast as proper Makefile would detect what can be skipped.
    • In reality the build takes 88 % of the time from the first build!
  • Is not reliable. Often, "git clean -xdf" is needed to get the build to pass.
  • It is hard to maintain and debug.

The proposal is to re-write the build system using current best practices and standard tools. Re-write from scratch is deemed to be more effecient than fixing bugs one by one because parts of current system are unnecessairly complex or can be replaced with standard solutions.

We forgot to create a ticket for this refactoring effort, so I've created one restrospectively and added all the relevant pull requests to it.

pr 132, master:

  • 4cd83fb51cc35a2ba7773b62a7aa8d295a1e1e4a Port all setup.py to setuptools
  • e12a70a8b1aa5daac4b8ab7ac681b09f664a8357 Remove ipapython/ipa.conf

pr171, master:

  • 329f080e6ac832ffd568e79ebca9907771f8ad61 Build: pass down LIBDIR definition from RPM SPEC to Makefile
  • 41da8732051c193166fa69cd9bc1152d39ad8720 Build: adjust include paths in daemons/ipa-kdb/tests/ipa_kdb_tests.c
  • 25dab77301f9e0289b94b0a672aed5067384c8ce Build: promote daemons/configure.ac to top-level configure.ac
  • b0cb6afa2308b9d96456f0355771ecbef0ca7263 Build: transform util directory to libutil convenience library
  • 6e1d777d2878de1e90e1dab4949075d31ffb0d52 Build: merge asn1/configure.ac to top-level configure.ac
  • 5e028b59bcd3c485cc034f491a0171f26b03ce3a Build: merge ipatests/man/configure.ac to top-level configure.ac
  • 927ddcb95aeb5c9bcc0cb08c5f08cecdccd0acb8 Build: merge install/configure.ac into top-level configure.ac
  • 0d37619db41abddabdd313f036f453821f438c8a Build: move translations from install/po/ to top-level po/
  • 881ab4edffa5e9c6c8bd8fb9762e04d776c4a573 Build: remove Transifex support
  • 0d7d6f3904287bb4903ffaa9d8c61e7593ae76d6 Build: merge client/configure.ac into top-level configure.ac
  • 1f1648691de6687e748c5b8267f43cf196bd7cc9 Build: modernize POPT library detection
  • fd3176a72951a2baece36620341991feb94b230a Build: modernize SASL library detection
  • b12da59a6cde3580f97e4ceb7303bc85857faf4d Build: modernize CURL library detection
  • 1e0143c159134337a00a91d4ae64e614f72da62e Build: modernize XMLRPC-client library detection
  • 8acb1f37f581e92a56951c069c990c77e83f3c42 Build: cleanup INI library detection
  • c8be979b3263723a9ab7b3c71efbe85b42d981e9 Build: use libutil convenience library for client
  • 1a4f287931edf0f3a76e97875f4af622716475af Build: use default error handling for PKG_CHECK_MODULES
  • 38628e46f05bde5ccc0fbb997f79364860713b48 Build: split API/ACI checks into separate Makefile targets
  • f25c0cb0c5f309944904fd32e781c27ae7e45a62 Build: split out egg-info Makefile target from version-update target
  • c954d0e1ba2a9ba8e8da679bc7246788d086d976 Build docs: update platform selection instructions
  • c70a2873f8a4447f8b38ad7b8468fc78c91bbb63 Build: replace ipaplatform magic with symlinks generated by configure
  • 3e79d8ad4aef1f62372a2169699df345348ba3e9 Docs: update docs about ipaplatform to match reality

pr159, master:

  • 21395d1724e6bf044438a8bc25ba028ed38cde8c spec file: clean up BuildRequires
  • 1077743d90902fc776f837f8486fa7f08b560673 spec file: do not include BuildRequires for lint by default
  • 0d370a959b6b55f995661b2febe55c379065c4d9 pylint: enable the import-error check
  • 0b91735c79a0ba577f9540e946180760a97913a4 ipaserver: remove ipalib import from setup.py
  • 9b534238157e003d2d7c3e1a7fd27531ab1dfd25 makeapi, makeaci: do not fail on missing imports
  • 9477e39b4b267922dbdd86a65869f773d980df8e client: remove unused libcurl build dependency
  • cc5ad6b3f951a6cb8298181690248d680c39922b pwpolicy: do not run klist on import

pr213, master:

  • c48e5fd811326dc64e19490f88003e442815a052 Build: move version handling from Makefile to configure
  • 0a17155e5b0434d4cab4d1696fac7f5ef88f0808 Build: replace hand-made Makefile with one generated by Automake
  • 81da45ffb13d126c9b56a2022d88ba8bed2ee18c Build: fix module name in ipaserver/setup.py
  • 8de11b091fc705f235b1304fb101c27a82dcda6f Build: Makefiles for Python packages
  • 3d6b8f8bdd5568c44d293cba960209941e4d2545 Build: fix ipaplatform detection for out-of-tree builds
  • 3a41b3bb8860cf73fef7efd54db2da5ecbd608d5 Build: fix make install in asn1 subdirectory
  • 820fd4c7ce6ccc80272f45d6f64227567692dd39 Build: respect --prefix for systemdsystemunitdir
  • 24feae47f26f40f757fbdd711399128d88c9b62c Build: fix Makefile.am files to separate source and build directories
  • b8d81ba3a12d93c38c4a0a8d439845746a32ae35 Build: remove non-existing README files from Makefile.am
  • fa8a468dba0ed866497669bd9c08b7de3a2cfbe3 Build: rename project from ipa-server to freeipa
  • 7282776c05c2fb254ae65b63977ba604be316038 Build: fix distdir target for translations
  • 2f6712893be5e66260a169c367a4607be6043d11 Build: fix man page distribution
  • 021a52d6801b74ded03cfdf6c7fb73bd1cab978f Build: fix distribution and installation of static files in top-level directory
  • f95098b2b645a62497dc6e1d66be2b7397567d25 Build: stop build when a step in web UI build fails
  • 441acf7797b2069e8d9a123aa11bb33fd42d9187 Build: fix distribution of static files for web UI
  • 24525fd086450616d4edd2aaf26dec868ff80ea9 Web UI: Remove offline version of Web UI
  • b910683e19356390351a6b82240762969ecf89c0 Build: fix distribution and installation of update LDIFs
  • 04be25082c60da01552d5e7c73d12930b10bd02e Build: fix distribution of libexec scripts
  • deec97abaec933709718464c4aa233a04de1844a Build: cleanup unused LDIFs from install/share
  • a125370becb045b6e757df88e520ef3f8ab4ca09 Build: Remove spurious EXTRA_DIST from install/share/Makefile.am
  • dabc65f6b1989fb8f938e4b7249fcf5d41706e17 Build: fix distribution of oddjob files
  • 886d9167eb939a3ab5226ca420c404a9810186cf Build: fix distribution of install/REDME.schema file
  • c951a491a9082b8b5931782f45f82e251eb93c3c Build: fix distribution of asn1/asn1c files
  • 0d5fe1dba0459b09bc7518d34c58444c96435801 Build: fix distribution of client header file
  • 125bf25577e58d11252cb41d34065d49f581e0ac Build: fix distribution of daemon/ipa-kdb files
  • 684a2c6a58b99a72f68e4c7f827d6601007cea26 Build: fix distribution of ipa-slapi-plugins/common files
  • 4fb2f535ca73dd16738ce4a3b692931fb26227aa Build: fix distribution of daemons/ipa-slapi-plugins/ipa-cldap files
  • 14bce67cf0cad1aecc132a2c67ad2dc686bcd2af Build: fix distribution of daemons/ipa-slapi-plugins/ipa-exdom-extop files
  • c1652f92af6bea13ecd96c0ad7be38784e2faeb5 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-otp-counter files
  • 278cda7ede3777f61f31ec77199d02954512e133 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-otp-lasttoken files
  • 53cd71a63c7d6ba97a5593e5a8922af71c5a4b6f Build: fix distribution of daemons/ipa-slapi-plugins/ipa-pwd-extop files
  • 74820fe3d8774244476357036406014680d54211 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-sidgen files
  • 39b17ef2abd885ab87c1a39d3036f762b6b084c8 Build: fix distribution of daemons/ipa-slapi-plugins/ipa-winsync files
  • f229bb56b73487758ed9bd9c7f0a4cc74134992b Build: fix distribution of daemons/ipa-slapi-plugins/topology files
  • 312e780041fc9025ca3c189e6c9fcb54c7340714 Build: integrate daemons/dnssec into build system
  • 8ffd3bdf142f0f852918186ce0a338a7818bbe8e Build: remove ancient checks/check-ra.py
  • d3cab75d7e79fbc89ef08df3e6d2b1e28b4ef163 Build: integrate contrib directory into build system
  • a027bf739848371fa91b5ba9766e031c9003d322 Build: remove init/SystemV directory
  • 288d624336d502a7df9856cdc2f6543b6e7c0b79 Build: integrate init and init/systemd into build system
  • 6cb0271509fe95ae38fc743f2a13faf32fe29a99 Build: create /var/run directories at install time
  • cc6382550fcf32bd4b843c922c10c5a5d247dd38 Build: distribute doc directory
  • dc5699a8a40dd27ffd25d9ad3185ba40d93ec95b Build: do not compress manual pages at install time
  • 4ce3aa3b12004ca4eb29e4bbca415a585fbd432f Build: remove unused dirs /var/cache/ipa/{sysupgrade,sysrestore} from SPEC
  • 75a944e980c64061e51f4ec7215033c118f39863 Build: fix KDC proxy installation and remove unused kdcproxy.conf
  • fee9bbd85afeac3593abd791de2d002bed300c8e Build: add make rpms target and convenience script makerpms.sh
  • 2df98772556de0d964028bbb78a9efbdd13ecd40 Build: remove obsolete instructions about BuildRequires from BUILD.txt
  • 14c1c8dfd0aa894af2d60dfa4f2ce2510d791328 Build: add Python lint target
  • f31a489d246e01250536b7187225fb7ca6398ba5 Build: add JS lint target
  • b54e9e86dfaed1320f7ccce560f82c233f67bf1a Build: add makeaci lint target
  • e3b537af18afa03b1f04530b42cdba5c1fc3ff97 Build: add makeapi lint target
  • 4498998f1763d673056423a73d3b3ff22f94954f Build: add polint target for i18n tests
  • c0674e89d1e6b5abd82cf3b7bf8054eec0fa6418 Build: fix make clean for web UI

master:

  • 9bb6d8643f4eb7214897de28821839a14a3bcb37 gitignore: ignore tar ball

master:

  • d20f6a5ef2467e780026f1040f5a11a7a77594ca Build: fix make clean to remove build artifacts from top-level directory

master:

  • 394edf5f055766fa0cdf70afab6d263f75d0d065 Build: remove unused and redundant code from configure.ac and po/Makefile.in
  • f6f5708a5ac56392f7a4b82f63c5e16cc1f1fd99 Build: IPA_VERSION_IS_GIT_SNAPSHOT checks if source directory is Git repo
  • 3dc5d2c6f9a0fc134616c615634ae505ef753f77 Build: use POSIX 1003.1-1988 (ustar) file format for tar archives
  • a691b7d1837595fecd37bf88a875cb0753f7e698 Build: IPA_VERSION_IS_GIT_SNAPSHOT re-generates version number on RPM build
  • 0023fb59240cb53a541763a89db038c186312154 Build: add make srpms target
  • 8c81c6c5b8ea62addbc175308a4e357c75d65ef0 Build: update IPA_VERSION_IS_GIT_SNAPSHOT to comply with PEP440
  • 961773bd0455e6fc723dbbed4f53e4b483360c32 Build: pass down %{release} from SPEC to configure

master:

  • 636aaa7dbc649e685233f382cb8dd424345bebd3 Build: move client directory handling from SPEC to Makefile.am
  • 20918579acb43391d5d04ee8050b37142a55df76 Build: move server directory handling from SPEC to Makefile.am
  • 1fa0ed954bb45b6e3858c1c54470b1d16ab204d9 Build: move web UI file installation from SPEC to Makefile.am
  • 5a5373464fa67289c9a178b1c0569f585dc6dc34 Build: document what should be in %install section of SPEC file

master:

  • 27e7a89a6289d0d3009f5f7feb9802b7db171a15 Build: workaround bug 1005235 related to Python paths in auto-generated Requires
  • 46b6b9e3091d08412912c37f46af497ddd0b8afb Build: support --enable-silent-rules for Python packages
  • 1cbd823990da0e931b666c4bc5c72f10d9de8115 Build: enable silent build in makerpms.sh
  • d5683726d290b71eb44ab3b3150381f062e74df1 Build: remove incorrect use of MAINTAINERCLEANFILES
  • e2060e8e5562ed6a4fe760eba1babb5c1761576a Build: update makerpms.sh to use same paths as rpmbuild
  • 6b9977f04199bf161d7171aedae9f97648c415c8 Build: fix file dependencies for make-css.sh

master:

  • 28c5e128c823f32787a4bde87e6b248928a2cd0a Build: ignore rpmbuild for lint target

master:

  • 4842231074683ff68be50b147560f5383aa305b6 Build: use standard infrastructure for translations
  • d40c376ccc1ec9292df0306134c7bfdfd096566e Build: support strip-po target for translations
  • 166257ec5b33dd2c95bbaf8463867c77fd6ef5db Build: clean in po/ removes *~ files as well
  • 9ef5a7de781f2508c2925225533973417458d0ea Build: automatically generate list of files to be translated in configure
  • 8a7962585069d7b0ff7e8d87ce094f07c16b3cd4 Build: integrate translation system tests again
  • 4c133837d149352a68e1d6cbefbb28e4ae048755 Build: remove ipa.pot from Git as it can be re-generated at any time

master:

  • 132b475c2586f3ced68724355e9c45722dccf604 Build: workaround bug while calling parallel make from rpmbuild

master:

  • e5686c0ccb4a71fa0ab460fd45280d9adba91cca Lowered the version of gettext

As far as I can tell the main refactoring work was done.
The last missing thing is client-only build which is tracked separately in ticket:6517.

master:

  • 1b85e59ceeb115dfb24e9c6bacb665b935f9543c spec file: do not define with_lint inside a comment

To be able to build rpms in an environment controlled by a FreeIPA domain we need to use a better tar format.
3dc5d2c6f9a0fc134616c615634ae505ef753f77 change configure to use the ustar format, but that is not enough as it limits UID/GID size to values less than 2 million.

master:

  • 2bc01ec5b4a91a805912bdada429a91ab08ed196 Use the tar Posix option for tarballs

master:

  • eb1f05d598d821f8e7eb5b8cfe606f570052f263 spec file: revert to the previous Release tag

Metadata Update from @pspacek:
- Issue assigned to pspacek
- Issue set to the milestone: FreeIPA 4.5

master:

  • 2a4f7f2cfaf6ac5ffaf4cc2b43fa0e9b5fa3ebe4 CONFIGURE: Improve detection of xmlrpc_c flags

Metadata Update from @mbasti:
- Issue close_status updated to: None (was: Fixed)
Metadata
None
None
None
Normal
refactoring
Build System
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.