Linux


I was tasked with a documentation project regarding use of yum repositories inside a customer’s environment and realized that I was having a hard time finding exactly the information I needed to build the full test environment required to test the documentation.  What will follow is a series of posts covering information I had to research as background.

Topics

  1. Background, Terminology and repository basics
  2. Setting up a corporate yum repository mirror for bandwidth and staged update management
  3. Setting up a private repository for additional software

Notes

As a quick note up front, based on what I’ve learned, I can’t find a good reason to include the software from a private repository in the update management repository – just build a new repository.

PXE Booting – there are instructions here which will break a PXE Boot environment. Our lab doesn’t PXE or Net boot, so we haven’t tested against them, but a colleague assures me it won’t work.

Bibliography

The following posts were used to build this series. None of them was 100% what I needed, and no source had everything I needed in a single location, but they were all useful, and in no order at all:
http://www.howtoforge.com/creating_a_local_yum_repository_centos
http://wiki.centos.org/HowTos/CreateLocalMirror
http://kenfallon.com/how-to-mirror-rhn-behind-your-firewall/
https://docs.fedoraproject.org/en-US/Fedora/14/html/Software_Management_Guide/ch08s04.html
http://www.outsidaz.org/blog/2012/02/19/using-reposync-to-provide-local-yumrpm-mirrors/

EDIT

There’s a new version of this post at https://www.totalnetsolutions.net/2017/09/13/kubuntu-17-04-on-dell-e5470-for-extreme-battery-life/, since it’s 4 years newer, you should head there instead.

/EDIT

It’s been a long time since I’ve updated my Linux buildout on this site.

I’ve recently upgraded to a Lenovo T430 (from a T500, and Dell before that). This laptop has the following hardware:

Jump to main sections with these links:
CPU Configuration
Network Configuration
Video/Bumblebee Configuration
Encryption / Security configuration
Battery saving configuration
Custom kernel .config
This is the first time in years I haven’t had a built-in 3G Modem for internet access, but with wifi tethering to my phone, I don’t think it’ll be an issue. Out of the box, everything I cared about worked. I didn’t test the fingerprint reader or the Nvidia graphics with the default install, and pretty quickly customized the system, but if you’re not into customization, rest assured, the Ubuntu team did a fantastic job on the setup.

CPU and Battery


As stated in this post from 2007, I’m a huge fan of extreme battery life. I’m still using cpufreqd, laptop-mode-tools, and checking their configuration with powertop to make sure I’m doing everything I can. I also custom-compile my kjernel, which I’ll discuss more below.

Kubuntu 12.04 defaults to the “ondemand” cpufreq driver, which is great for power savings, except that it does its speed modulation in preference of performance. That is, when the lowest speed of the processor (1.2GHz in my case) isn’t enough, ondemand immediately jumps the CPU speed to the fastest available (2.9GHz in my case). Then, when the fastest is more than required, ondemand steps off slowly. This is perfect for video and gaming applications, and most people. I, however, greatly prefer the “conservative” driver, which works in the opposite manner: when an application needs more power, the conservative driver steps up 1 CPU speed level at a time, until the appropriate CPU speed is reached. Then, when the utilization drops off, conservative immediately drops all the way to the slowest speed, to step back up again if needed.
Using cpufreqd allows me to control this even more granularily, while not getting in the way of the kernel modules. My configuration uses “ondemand” when I’m plugged in, and “conservative” when I’m on battery power. The small delay in performance is worth the added minutes in battery life, especially since most of my on-battery time is very low-demand applications.
Since sometime in 2011, Ubuntu, however, has not shipped a working cpufreqd daemon – it’s apparently broken in the upstream Debian as well, and is well documented in this Launchpad bug. So I downloaded the cpufreqd-dev source package, the patch, and rebuilt cpufreqd myself. Now that it’s working, I can use the attached updated cpufreqd.conf configuration.


Network

Both the ethernet and wireless adapters work right on first install with Kubuntu 12.10.  The wireless uses the iwlwifi driver, and connects to my router at 104mbps.  I’m still using the script from this post to get my MTU set to 9000, rather than 1500, for jumbo frames support when in the main office network. This has a major effect on network speed for large transfers, but most networks don’t support it.h

The Wireless adapter needed no changes, and NetworkManager handles it beautifully, even when tethering to my phone.

Sound


I haven’t had any sound issues, but a few users have reported problems when using the docking station. According to this post on ThinkPad Forums, the solution is to simply edit /etc/modprobe.d/alsa-base.conf and add:
options snd-hda-intel model=thinkpad
This causes no issues on my system.

Video


This laptop ships with two video cards, an Intel (which uses the kernel i915 driver) low-power adapter, and an Nvideo high-performance, high-power adapter. In Windows, you can click an application to switch between the two for all applications. The Nvidia adapter uses about 10W more power than the Intel card, which means using the Nvidia adapter alone halves the system’s battery life.
By default, the Ubuntu kernel enables a feature called “vgaswitcheroo” which is well documented on the Ubuntu help site. I had a hard time getting it to work with my custom kernel, though, even though it was enabled. KDE and lightdm just didn’t want to swtich the laptop panel over to the Nvidia card. This *may* have something to do with my BIOS settings, which I currently can’t change due to office IT restrictions.
There is a new project called Bumblebee, which allows the user to use the Intel card, and only turn the Nvidia on for some applications. This gives the best of both worlds for power and battery savings, but is a work-in-progress, and not all applications run under Bumblebee. I’m using the Primus additions. Installation of Bumblebee is documented here, and Primus installation is documented here. I didn’t have to make any changes to get these installs working with simple applications.


VMware Workstation 9

VMware Workstation 9, however, offered some interesting challenges with Bumblebee. Out of the box, Vmware Workstation 9 installed (even with my custom kernel), and ran great, but would always give a warning that 3D acceleration was not available, which I expected while using the Intel card. However, Bumblebee has some limitations which mean it can’t run VMware workstation by default. I wrote a script to handle this, which I wrote up last night. I’m using a lot of work of others, so follow the links on that page to cmillersp’s post on the VMware Community Forums.


Encryption and Security

During installation, I chose the option to use an encrypted LVM volume. This uses DM-Crypt to encrypt the full HDD, so that it has to be unlocked at boot time. The Kubuntu installer seems to forget this fact, so it also asks you to set up ecryptfs private home directories, which is NOT neccessary for a single-user laptop, since the whole OS is already encrypted. The only oddity with dm-crypt is that sometimes the splash screen prompt to unlock the computer doesn’t show. If I just wait for disk activity to disappear, and have a blank screen, I can just type the passphrase, and it’ll still unlock successfully.

I don’t have the fingerprint reader set up, but if I do, I’ll update this post.


Battery and Power Savings

First, I use the configuration in the CPU Configuration section above for cpufreqd. Then I use laptop-mode-tools to set other configuration settings. I’ve attached all the files I have modified here, and it’s fairly power-saving aggressive. The only thing I should do, but don’t, is to disable the bluetooth adapter when I’m on battery, since it uses 1W just for the adapter. However, I have bluetooth headphones and a bluetooth mouse, which is why I have the bluetooth adapter in the laptop to begin with, so disabling it removes some critical functionality. I DO have it set to autosuspend, which is a little annoying when I go back to the mouse after 5 seconds of inactivity, but the annoyance is worth the savings, especially when I’m writing a long post like this.
laptop-mode.conf
All of these go in /etc/laptop-mode/conf.d :
auto-hibernate.conf
cpufreq.conf
ethernet.conf
intel-hda-powersave.conf
video-out.conf

With these settings, I was able to write this whole post, with the bluetooth mouse connected, running at an average of 13.2W. In 3.4 hours (I was doing other tasks, including feeding and changing my napping baby), I used 41% of my 93.6Wh battery. If I were to take this on a plane, I’d kill the wireless and bluetooth for probably another 2W savings, but I’d do that by hand.


Kernel Configuration

I have been building a custom kernel for my laptop for about 6 years now. The default Ubuntu image uses “generic-x86_64” for the processor family, but all of my laptops are “Core2 / Newer Xeon”. Just making that single change to my kernel results in about 0.5W-1W less power consumption, due to the increased efficiency gained by the kernel knowing about the new processor registers and commands that aren’t available to older processors. This greater CPU efficiency also means lower temperatures, and therefore lower fan speed.

My kernel configuration is attached here. Build it by following the instructions at the Ubuntu Help site

sudo apt-get install linux-kernel-devel fakeroot kernel-wedge build-essential
sudo apt-get install linux-source
sudo usermod -a -G src YOUR_USERNAME

Now log out and back in, so that you’re a member of the “src” group.

cd /usr/src
sudo chown -R $USER:src .
tar -jxf ./linux-source-3.5.0/linux-source-3.5.0.tar.bz2
ln -s linux-source-3.5.0 linux
cd linux
wget http://www.totalnetsolutions.net/wp-content/uploads/2012/12/rob-config-20121204c.txt
mv rob-config-20121204c.txt .config
make oldconfig
make menuconfig

Make any changes you want in here, then exit and save

fakeroot make-kpkg --initrd --append-to-version=.20121204c kernel_image kernel_headers

You’ll get 2 DEB files in /usr/src that you can then install and boot to. the “append-to-version” I use as a dating system for my kernels. “20121204c” means the 4th kernel attempt on December 4th, 2012, the day I recieved this laptop.

All Modified Files


Laptop-mode-tools config:
laptop-mode.conf
All of these go in /etc/laptop-mode/conf.d :
auto-hibernate.conf
cpufreq.conf
ethernet.conf
intel-hda-powersave.conf
video-out.conf
Other configs:
custom kernel .config.
cpufreqd.conf
Thanks.

I have a new Lenovo T430 with Nvidia/Intel hybrid graphics. The Intel card is for power saving, the Nvidia card for performance. The easiest way to handle this setup is to just choose the correct card on bootup, but this is inconvenient. Bumblebee provides a way to use the Intel card most of the time, but use the Nvidia for high-performance tasks such as games, and ONLY for those apps. Once it’s set up, it works pretty well, except for a few apps that require additional tweaking, such as Steam, or Wine.

One of the apps that requires tweaking is VMware Workstation (I’m running WKS 9). cmillersp provided a great write-up on the VMware Communities, which is what I set up on my system. VMware Workstation runs great – I can load OpenGL 3D apps in VMs and everything runs fantastic.

Except that I run VMs all the time, and using the Nvidia card all the time kills the performance benefit of the Intel card twice: once because I’m using the Nvidia card, and once again because BOTH GPUs are running at the same time. So I wanted a way to dynamically choose which card to run VMware under, based on whether I was on AC power or battery power.

The result is the attached script below, which I’ll be submitting to the Bumblebee wiki / project, as well as the VMware forums. This is version 1, which works as follows:

  1. Must be installed by “sudo ~/bin/vmware –install” – it will make the cd /usr/lib/vmware/bin; mv vmware-vmx vmware-vmx.real and then do the script creation at /usr/lib/vmware/bin/vmware-vmx, so that it’s a single portable script.
  2. Takes several options to force using the Nvidia card or not
  3. If no force option is applied, automatically determines AC adapter state or battery charge/discharge state
  4. Based on the above, decides whether to launch /usr/bin/vmware normally, or via the instructions from cmillersp

I’ve done some additional work to make it run via either “primusrun” or “optirun”. Optirun is much slower, but at least functional with fewer installs. I hope this is useful to someone else!

Edit: 2012-12-17 – v1.1 added gksu auto-detection
Edit: 2014-01-08 – Moved to github repo: github.com/docsmooth/vmware-bumblebee

I was troubleshooting a problem with some other vendor’s software tonight on a Red Hat Enterprise Linux 5.3 system.  We were able to reproduce the problem in the lab,  which was a huge boost to production, and insight, but we hit a wall when we got this error:

GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-42.el5)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /opt/vendor/redacted/process...(no debugging symbols found)...done.
Attaching to program: /opt/vendor/redacted/process, process 16492
ptrace: Operation not permitted.

The weird thing is that we were running gdb as root, and it was 2.6.18. In the latest Ubuntu versions, a security hardening option has been added to the kernel to limit gdb (profiling, particularly, which gdb requires) to only being run on child processes. Since this was Red Hat Enterprise Linux 5.3, it didn’t have this option.

Well, it turns out that explaining it to #gdb on Freenode pointed us to the solution: a parent process had been attached to via “strace -f”. Since only one profiling process can run on any program at one time, the parent process’s “strace”, by following all forks with the “-f”, blocked out our “gdb” from attaching to the child. Simply adding 1 line:

pkill strace
gdb /opt/vendor/redacted/process `pgrep process`

to our debugging solved the mysterious “ptrace: Operation not permitted.” error which was showing me no results in web searches. FYI: this absolutely will block gcore in the same way.

We have a RHEL 5 box running some POE components that require the XS version of Scalar::Util.  However, the yum repos appear to have the non-XS version included, so every upgrade of perl breaks the Perl code that requires Scalar::Util.
The fix is here:

http://www.cybersprocket.com/2011/programming-languages/weaken-is-only-available-with-the-xs-version-of-scalarutil/

cpan
> force install Scalar::Util

But why is this? If you check out the “Diagnostics” section for Scalar::Util at CPAN, you’ll see:

NAME is only available with the XS version of Scalar::Util

Scalar::Util contains both perl and C implementations of many of its functions so that those without access to a C compiler may still use it. However some of the functions are only available when a C compiler was available to compile the XS version of the extension

Meaning – the update from yum must install the perl-only impelementation for cross-compatibility. What’s of most interest here is that “force install Scalar::Util” isn’t actually the full answer – you have to have gcc installed as well, so that there’s a C-compiler available during the CPAN forced reinstall. In case your production systems don’t have gcc installed already.

« Previous PageNext Page »