Some python packages are not being built

[marsclic]

A few days ago I noticed that some python packages in my PC at work ( a Core2 x86_64 ) are not building anymore. My home PC however does not show the same problem ( a Dual-dual Opteron x86_64 as well).

I attached the output of trying to build the pyrex package, but many other packages show the same problem. Please notice the "python." invocation which is the source of the error: ============================ ...

Done builtin_loadenv

Starting src_compile

• Building of dev-python/pyrex-0.9.8.5-r1 with Python ....

python. setup.py build -b build-. /var/tmp/paludis/dev-python-pyrex-0.9.8.5-r1/temp/loadsaveenv: line 1209: python.: command not found

!!! ERROR in dev-python/pyrex-0.9.8.5-r1: !!! In python_execute_function at line 4050 !!! Building failed with Python . in build_modules() function

!!! Call stack: !!! * python_execute_function (/var/tmp/paludis/dev-python-pyrex-0.9.8.5-r1/temp/loadsaveenv:4050) !!! * distutils_src_compile (/var/tmp/paludis/dev-python-pyrex-0.9.8.5-r1/temp/loadsaveenv:1211) !!! * src_compile (/var/tmp/paludis/dev-python-pyrex-0.9.8.5-r1/temp/loadsaveenv:4375) !!! * ebuild_f_compile (/usr/libexec/paludis/2/src_compile.bash:56) !!! * ebuild_main (/usr/libexec/paludis/ebuild.bash:575) !!! * main (/usr/libexec/paludis/ebuild.bash:591)

diefunc: making ebuild PID 7916 exit with error die trap: exiting with error.

Install error:

• In program paludis -i pyrex:
• When performing install action from command line:
• When executing install task:
• When installing 'dev-python/pyrex-0.9.8.5-r1:0::gentoo':
• When running an ebuild command on 'dev-python/pyrex-0.9.8.5-r1:0::gentoo':
• Install error: Install failed for 'dev-python/pyrex-0.9.8.5-r1:0::gentoo'

==================================

I have tried downgrading and recompiling paludis and several other packages which I thought were related to no avail. Below is a few more bits of information:

# eselect python list Available python interpreters: [1] python2.5 [2] python2.6 * [3] python3.1

#python Python 2.6.2 (r262:71600, Aug 10 2009, 11:18:48) [GCC 4.3.4]

#ls -l /usr/bin/python* lrwxrwxrwx 1 root root 14 2009-08-10 11:20 /usr/bin/python -> python-wrapper lrwxrwxrwx 1 root root 9 2009-08-10 11:20 /usr/bin/python2 -> python2.6 -rwxr-xr-x 1 root root 6568 2009-07-23 14:33 /usr/bin/python2.5 -rwxr-xr-x 1 root root 6560 2009-08-10 11:20 /usr/bin/python2.6 lrwxrwxrwx 1 root root 9 2009-08-09 18:13 /usr/bin/python3 -> python3.1 -rwxr-xr-x 1 root root 10728 2009-08-09 18:13 /usr/bin/python3.1

It doesn't matter if I eselect 2.5 or 2.6, the problem persists. I even tried rebuilding "system", but that didn't help either. Instead, I found that these packages are broken as well:

• sys-apps/file-5.03:0::gentoo: failure
• dev-java/java-config-2.1.8-r1:2::gentoo: failure
• sys-libs/db-4.7.25_p4:4.7::gentoo: failure
• dev-python/pyrex-0.9.8.5-r1:0::gentoo: failure
• dev-python/dbus-python-0.83.0-r1:0::gentoo: skipped (dependency '>=dev-python/pyrex-0.9.3-r2' unsatisfied)
• dev-python/pycairo-1.8.6:0::gentoo: failure
• sys-libs/cracklib-2.8.13-r1:0::gentoo:failure
• dev-python/numpy-1.3.0:0::gentoo:failure
• dev-libs/cyrus-sasl-2.1.23:2::gentoo: failure

Here is the output of paludis --info: paludis --info dev-python/pyrex paludis@1250098246: [WARNING e.ebuild.configuration.master_repository] In thread ID '27457':

... In program paludis --info dev-python/pyrex: ... When making environment from specification: ... When loading paludisenvironment: ... When creating repository from file '/etc/paludis/repositories/kdetest.conf: ... When making ebuild repository from repo_file '/etc/paludis/repositories/kdetest.conf': ... Key 'master_repository' in '/etc/paludis/repositories/kdetest.conf' will override '/var/paludis/repositories/kdetest/metadata/layout.conf'.

paludis 0.38.2 Paludis build information:

Compiler:

CXX: x86_64-pc-linux-gnu-g++ 4.3.4 CXXFLAGS: -march=core2 -O2 -pipe LDFLAGS: -Wl,-O1 DATE: 2009-08-10T16:58:35-0700

Libraries:

C++ Library: GNU libstdc++ 20090804

Paths:

DATADIR: /usr/share LIBDIR: /usr/lib64 LIBEXECDIR: /usr/libexec SYSCONFDIR: /etc PYTHONINSTALLDIR: /usr/lib64/python2.6/site-packages RUBYINSTALLDIR: /usr/lib64/ruby/site_ruby/1.8/x86_64-linux

System:

Linux marius 2.6.30-gentoo-r4 #1 SMP PREEMPT Mon Jul 27 20:10:05 PDT 2009 x86_64 Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz GenuineIntel? GNU/Linux

Reduced Privs:

reduced_uid: 109 reduced_uid->name: paludisbuild reduced_uid->dir: /var/tmp/paludis reduced_gid: 1001 reduced_gid->name: paludisbuild

Environment:

Format: paludis Config dir: /etc/paludis World file: /var/db/pkg/world

Repository installed-virtuals:

format: installed_virtuals root: /

Repository virtuals:

format: virtuals

Repository gentoo:

format: ebuild location: /usr/portage append_repository_name_to_write_cache: true binary_destination: false binary_keywords: binary_uri_prefix: builddir: /var/tmp/paludis cache: /usr/portage/metadata/cache distdir: /usr/portage/distfiles eapi_when_unknown: 0 eapi_when_unspecified: 0 eclassdirs: /usr/portage/eclass ignore_deprecated_profiles: false layout: traditional names_cache: /var/cache/names newsdir: /usr/portage/metadata/news profile_eapi_when_unspecified: 0 profiles: /etc/make.profile securitydir: /usr/portage/metadata/glsa setsdir: /usr/portage/sets sync: rsync://rsync.gentoo.org/gentoo-portage sync_options: use_manifest: use write_cache: /var/cache/paludis/metadata

Package information:

app-admin/eselect-compiler: (none) app-shells/bash: 4.0_p28 dev-java/java-config: 1.3.7-r1 2.1.8-r1 dev-lang/python: 2.5.4-r3 2.6.2-r1 3.1 dev-python/pycrypto: 2.0.1-r8 dev-util/ccache: 2.4-r8 dev-util/cmake: 2.6.4-r2 dev-util/confcache: (none) sys-apps/baselayout: 2.0.1 sys-apps/openrc: 0.4.3-r3 sys-apps/sandbox: 2.0 sys-devel/autoconf: 2.13 2.63-r1 sys-devel/automake: 1.10.2 1.11 1.5 1.6.3 1.7.9-r1 1.8.5-r3 1.9.6-r2 sys-devel/binutils: 2.19.1-r1 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.30-r1 (for sys-kernel/linux-headers::installed)

Repository installed:

format: vdb location: /var/db/pkg builddir: /var/tmp/paludis eapi_when_unknown: 0 names_cache: /var/cache/paludis/names provides_cache: /var/cache/paludis/provides root: /

Repository kde:

format: ebuild location: /var/paludis/repositories/kdetest append_repository_name_to_write_cache: true binary_destination: false binary_keywords: binary_uri_prefix: builddir: /var/tmp/paludis cache: /var/empty distdir: /usr/portage/distfiles eapi_when_unknown: 0 eapi_when_unspecified: 0 eclassdirs: /usr/portage/eclass /var/paludis/repositories/kdetest/eclass ignore_deprecated_profiles: false layout: traditional master_repository: gentoo names_cache: /var/paludis/repositories/kdetest/.cache/names newsdir: /var/paludis/repositories/kdetest/metadata/news profile_eapi_when_unspecified: 0 profiles: /etc/make.profile securitydir: /var/paludis/repositories/kdetest/metadata/glsa setsdir: /var/paludis/repositories/kdetest/sets sync:  git://git.overlays.gentoo.org/proj/kde.git sync_options: use_manifest: use write_cache: /var/cache/paludis/metadata

Repository marius:

format: ebuild location: /usr/local/portage/marius append_repository_name_to_write_cache: true binary_destination: false binary_keywords: binary_uri_prefix: builddir: /var/tmp/paludis cache: /var/empty distdir: /usr/portage/distfiles eapi_when_unknown: 0 eapi_when_unspecified: 0 eclassdirs: /usr/portage/eclass /usr/local/portage/marius/eclass ignore_deprecated_profiles: false layout: traditional master_repository: gentoo names_cache: /var/cache/names newsdir: /usr/local/portage/marius/metadata/news profile_eapi_when_unspecified: 0 profiles: /etc/make.profile securitydir: /usr/local/portage/marius/metadata/glsa setsdir: /usr/local/portage/marius/sets sync: sync_options: use_manifest: use write_cache: /var/cache/paludis/metadata

Repository pioto-overlay:

format: ebuild location: /var/paludis/repositories/pioto-overlay append_repository_name_to_write_cache: true binary_destination: false binary_keywords: binary_uri_prefix: builddir: /var/tmp/paludis cache: /var/empty distdir: /usr/portage/distfiles eapi_when_unknown: 0 eapi_when_unspecified: 0 eclassdirs: /usr/portage/eclass /var/paludis/repositories/pioto-overlay/eclass ignore_deprecated_profiles: false layout: traditional master_repository: gentoo names_cache: /var/paludis/repositories/pioto-overlay/.cache/names newsdir: /var/paludis/repositories/pioto-overlay/metadata/news profile_eapi_when_unspecified: 0 profiles: /etc/make.profile securitydir: /var/paludis/repositories/pioto-overlay/metadata/glsa setsdir: /var/paludis/repositories/pioto-overlay/sets sync:  git://git.pioto.org/pioto-overlay.git sync_options: use_manifest: use write_cache: /var/cache/paludis/metadata

Package dev-python/pyrex-0.9.8.5-r1:0::gentoo:

paludis@1250098246: [WARNING e.ebuild.userpriv_disabled] In thread ID '27457':

... In program paludis --info dev-python/pyrex: ... When handling query 'dev-python/pyrex': ... When infoing 'dev-python/pyrex-0.9.8.5-r1:0::gentoo': ... When checking permissions on '/var/tmp/paludis' for userpriv: ... Directory '/var/tmp/paludis' owned by group 'root', not 'paludisbuild', so cannot enable userpriv

Running ebuild phase killold as paludisbuild:paludisbuild... Starting builtin_killold Done builtin_killold Completed ebuild phase killold Running ebuild phases initmisc infovars as paludisbuild:paludisbuild... Starting builtin_initmisc Done builtin_initmisc Starting builtin_infovars

ACCEPT_KEYWORDS=amd64 CBUILD=x86_64-pc-linux-gnu CFLAGS=-march=core2 -O2 -pipe CHOST=x86_64-pc-linux-gnu CONFIG_PROTECT= CONFIG_PROTECT_MASK= CPPFLAGS= CTARGET= CXXFLAGS=-march=core2 -O2 -pipe DISTDIR=/usr/portage/distfiles FEATURES=strict sfperms FFLAGS= GENTOO_MIRRORS= INSTALL_MASK= LANG= LC_ALL=C LDFLAGS=-Wl,-O1 LINGUAS=en_US MAKEOPTS=-j5 PORTAGE_COMPRESS= PORTAGE_COMPRESS_FLAGS= PORTAGE_CONFIGROOT= PORTAGE_RSYNC_EXTRA_OPTS= PORTAGE_RSYNC_OPTS= PORTAGE_TMPDIR=/var/tmp/paludis PORTDIR=/usr/portage PORTDIR_OVERLAY= SYNC= USE=amd64 alsa_cards_ali5451 alsa_cards_als4000 alsa_cards_atiixp alsa_cards_atiixp-modem alsa_cards_bt87x alsa_cards_ca0106 alsa_cards_cmipci alsa_cards_emu10k1x alsa_cards_ens1370 alsa_cards_ens1371 alsa_cards_es1938 alsa_cards_es1968 alsa_cards_fm801 alsa_cards_hda-intel alsa_cards_intel8x0 alsa_cards_intel8x0m alsa_cards_maestro3 alsa_cards_trident alsa_cards_usb-audio alsa_cards_via82xx alsa_cards_via82xx-modem alsa_cards_ymfpci alsa_pcm_plugins_adpcm alsa_pcm_plugins_alaw alsa_pcm_plugins_asym alsa_pcm_plugins_copy alsa_pcm_plugins_dmix alsa_pcm_plugins_dshare alsa_pcm_plugins_dsnoop alsa_pcm_plugins_empty alsa_pcm_plugins_extplug alsa_pcm_plugins_file alsa_pcm_plugins_hooks alsa_pcm_plugins_iec958 alsa_pcm_plugins_ioplug alsa_pcm_plugins_ladspa alsa_pcm_plugins_lfloat alsa_pcm_plugins_linear alsa_pcm_plugins_meter alsa_pcm_plugins_mmap_emul alsa_pcm_plugins_mulaw alsa_pcm_plugins_multi alsa_pcm_plugins_null alsa_pcm_plugins_plug alsa_pcm_plugins_rate alsa_pcm_plugins_route alsa_pcm_plugins_share alsa_pcm_plugins_shm alsa_pcm_plugins_softvol apache2_modules_actions apache2_modules_alias apache2_modules_auth_basic apache2_modules_authn_alias apache2_modules_authn_anon apache2_modules_authn_dbm apache2_modules_authn_default apache2_modules_authn_file apache2_modules_authz_dbm apache2_modules_authz_default apache2_modules_authz_groupfile apache2_modules_authz_host apache2_modules_authz_owner apache2_modules_authz_user apache2_modules_autoindex apache2_modules_cache apache2_modules_dav apache2_modules_dav_fs apache2_modules_dav_lock apache2_modules_deflate apache2_modules_dir apache2_modules_disk_cache apache2_modules_env apache2_modules_expires apache2_modules_ext_filter apache2_modules_file_cache apache2_modules_filter apache2_modules_headers apache2_modules_include apache2_modules_info apache2_modules_log_config apache2_modules_logio apache2_modules_mem_cache apache2_modules_mime apache2_modules_mime_magic apache2_modules_negotiation apache2_modules_rewrite apache2_modules_setenvif apache2_modules_speling apache2_modules_status apache2_modules_unique_id apache2_modules_userdir apache2_modules_usertrack apache2_modules_vhost_alias elibc_glibc input_devices_evdev input_devices_keyboard input_devices_mouse kernel_linux lcd_devices_bayrad lcd_devices_cfontz lcd_devices_cfontz633 lcd_devices_glk lcd_devices_hd44780 lcd_devices_lb216 lcd_devices_lcdm001 lcd_devices_mtxorb lcd_devices_ncurses lcd_devices_text linguas_en_US qemu_softmmu_targets_i386 qemu_softmmu_targets_x86_64 qemu_user_targets_i386 qemu_user_targets_x86_64 userland_GNU video_cards_nvidia amd64

Done builtin_infovars Completed ebuild phases initmisc infovars Running ebuild phase tidyup as paludisbuild:paludisbuild... Starting builtin_tidyup

rm -fr /var/tmp/paludis/dev-python-pyrex-0.9.8.5-r1-info

Done builtin_tidyup Completed ebuild phase tidyup

Any help would be appreciated, thanks!

[ciaranm]

Building of dev-python/pyrex-0.9.8.5-r1 with Python

Why's it saying that? It doesn't say that for me and I can't see anything that'd cause that in the eclasses or ebuilds.

[marsclic]

I don't know, but I'm trying to find out. The problem is not specific with pyrex, though - there are many others in the same situation, such as setuptools. Here are some more data points: earlier versions of pyrex, and others such as setuptools, do not show the "python." problem but the build fails anyway:

=================== (1 of 1) Installing dev-python/pyrex-0.9.8.5:0::gentoo

paludis@1250112518: [WARNING e.ebuild.userpriv_disabled] In thread ID '2455':

... In program paludis -i =dev-python/pyrex-0.9.8.5: ... When performing install action from command line: ... When executing install task: ... When fetching 'dev-python/pyrex-0.9.8.5:0::gentoo': ... When checking permissions on '/usr/portage/distfiles' for userpriv: ... Directory '/usr/portage/distfiles' owned by group 'portage', not 'paludisbuild', so cannot enable userpriv

Checking 'Pyrex-0.9.8.5.tar.gz'... ok paludis@1250112518: [WARNING e.ebuild.userpriv_disabled] In thread ID '2455':

... In program paludis -i =dev-python/pyrex-0.9.8.5: ... When performing install action from command line: ... When executing install task: ... When installing 'dev-python/pyrex-0.9.8.5:0::gentoo': ... When checking permissions on '/usr/portage/distfiles' for userpriv: ... Directory '/usr/portage/distfiles' owned by group 'portage', not 'paludisbuild', so cannot enable userpriv

Running ebuild phase killold as root:root... Starting builtin_killold Done builtin_killold Completed ebuild phase killold Running ebuild phases init saveenv as root:root... Starting builtin_init Done builtin_init Starting builtin_saveenv Done builtin_saveenv Completed ebuild phases init saveenv

--- No need to do anything for setup phase

Running ebuild phases loadenv unpack saveenv as root:root... Starting builtin_loadenv Done builtin_loadenv Starting src_unpack Unpacking Pyrex-0.9.8.5.tar.gz to /var/tmp/paludis/dev-python-pyrex-0.9.8.5/work

tar zxf /usr/portage/distfiles/Pyrex-0.9.8.5.tar.gz --no-same-owner

Done src_unpack Starting builtin_saveenv Done builtin_saveenv Completed ebuild phases loadenv unpack saveenv Running ebuild phases loadenv compile saveenv as root:root... Starting builtin_loadenv Done builtin_loadenv Starting src_compile

python setup.py build

!!! ERROR in dev-python/pyrex-0.9.8.5: !!! In distutils_src_compile at line 1202 !!! Building failed

!!! Call stack: !!! * distutils_src_compile (/var/tmp/paludis/dev-python-pyrex-0.9.8.5/temp/loadsaveenv:1202) !!! * src_compile (/var/tmp/paludis/dev-python-pyrex-0.9.8.5/temp/loadsaveenv:4312) !!! * ebuild_f_compile (/usr/libexec/paludis/0/src_compile.bash:51) !!! * ebuild_main (/usr/libexec/paludis/ebuild.bash:575) !!! * main (/usr/libexec/paludis/ebuild.bash:591)

diefunc: making ebuild PID 2728 exit with error die trap: exiting with error.

Install error:

• In program paludis -i =dev-python/pyrex-0.9.8.5:
• When performing install action from command line:
• When executing install task:
• When installing 'dev-python/pyrex-0.9.8.5:0::gentoo':
• When running an ebuild command on 'dev-python/pyrex-0.9.8.5:0::gentoo':
• Install error: Install failed for 'dev-python/pyrex-0.9.8.5:0::gentoo'

==========================

[ingmar]

Replying to ciaranm:

Building of dev-python/pyrex-0.9.8.5-r1 with Python

Why's it saying that? It doesn't say that for me and I can't see anything that'd cause that in the eclasses or ebuilds.

That's distutiles.eclass, & python.eclass, specifically python_execute_function()

I'm going to bet that you're running into the broken python wrapper that hit ~arch in beginning of august. See  http://bugs.gentoo.org/279915

In any case, unlikely to be a paludis bug.

[marsclic]

Hi ingmar, no I don't think it's related to the same eselect-python bug. I have eselect-python-20090804, which supposedly fixed the problems in the bug you mentioned. I even applied the patch from bug 280780, even though I had little hope of that being the fix. It didn't help with my problem. My PC at home has a similar configuration but has no problems at all.

[marsclic]

Ok, I'm not really an eclass(y) guy, but I have tried some experiments. I hard-coded python.eclass to just call "python" (which now is the python-wrapper thing) and it stopped showing the "python." invocation. However, now the problem is at distutils.eclass, where the function building() breaks. Taking a look at it, it seems that PYTHON_ABI is a ".", and $@ is empty. It's as if it was not being passed any parameters. It seems the loadsaveenv script that is being generated by paludis is not being able to determine the python environment correctly. I hope that helps a bit. BTW, I'm running the latest eselect (1.1.13) and eselect-python (20090814) now.

[ciaranm]

Unfortunately you're the only person who can reproduce this, so you'll need to track down why the abi var is wrong yourself.

[marsclic]

Well, I will keep you posted. So far I have no idea what's causing this. My PC at home has a very similar configuration and is healthy. It's only my box at work that is doing this.

[marsclic]

This is really strange. I changed line 63 of python.eclass from: python=${python:-/usr/bin/python}

to:

python=/usr/bin/python

and everything started building again. It seems that the shell that paludis is using when interpreting loadsaveenv has problems evaluating that expression. Any idea what could be causing this?

[ciaranm]

Ok, so next you need to find out where the python env var is being set. You might want to try shoving a 'set -x' in ebuild.bash from Paludis and then grepping for 'python='.

[marsclic]

Ok, problem solved. Had nothing to do with paludis directly. There was a rogue </bin/python> link to </usr/bin/python> that some packages would pick up instead of </usr/bin/python>. It appears that python-wrapper doesn't work if anything is linked to it from outside of /usr/bin. Strange that 'which python' would return only </usr/bin/python>, and that's why it took me a while to figure that one out.

Thanks for the help!