Archive Page 2


Setting up a corporate yum repository mirror for bandwidth and staged update management

Part 2 of the seriesYum Repositories for Corporate Deployments

Part 1 of this series is: Yum Repositories: Background, Terminology and basics.

The most basic setup of a corporate yum repository would be to build a “base” repository for building centralized installs of systems off of the netinstall CD/USB stick provided by your Linux distro. The next basic repository to set up would be an “updates” repository, synced back to the distro, for bandwidth management for scheduled updates. We’ll walk through these 2 setups in order.

Setting up the repository host server

The first thing to determine is *how* you’re going to host your repository.  HTTP is probably the most common, but FTP and NFS repositories can be built as well.  The additional work for an NFS repository is minimal, so we’ll cover that as well as HTTP, using apache 2.2 as the HTTP server.

First, install the NFS and HTTP server components as well as the required reposync utility (this information is exactly the same for an NFS or HTTP apt repository for Ubuntu – a post for another time) and set them to autostart:
sudo /usr/bin/yum install nfs httpd yum-utils
sudo /sbin/chkconfig on http
sudo /sbin/chkconfig on nfs

Next set up the apache server to share the repository. This requires deciding *where* to host the repository, so we’ll use /srv/repo as our base. Create a file /etc/httpd/conf.d/repo.conf with these contents:

Alias /centos /srv/repo

<directory /srv/repo>
Options +Indexes
AllowOverride None
order allow,deny
allow from all

And now share out that directory via NFS and restart the servers:

echo '/srv/repo *(ro,async)' >> /etc/exports
service nfs restart
service httpd restart

The last suggestion I’d make is to create a subdirectory for each OS major version and its architectures in the root of your repository heirarchy, which is simple with bash expansion:

mkdir -p /srv/repo/{5,6}

Setting up a “base” repository

This is as simple as copying the contents of the install DVDs or CDs into the repository. We’ll use the CentOS materials for these examples, but Red Hat Enterprise Linux and Scientific Linux work the same.
First, we need to create our repository structure. This walkthrough will only demo RHEL 6 derivatives, but 5.x will work the same. This is the first time to think about which repositories to build. I’ll create 3 total: “base”, “updates” and “internal”. “base” and “updates” will be the distrobution-provided repositories, and “internal” will be for our company’s internal software. The “base” repository actually lives in a folder called “os”, and each repository has subfolders for each architecture that it supports. Therefore, we need to create 3 directories in /srv/repo/6: “os”, “updates”, and “internal”, and each of those gets 2 subdirectories for our supported architectures: “i386” and “x86_64”. (Also simple with bash expansion.)

mkdir -p /srv/repo/6/{os,updates,internal}/{i386,x86_64}

Now copy the files:

sudo mount -t auto -o loop,ro CentOS-6.3-x86_64-bin-DVD1.iso /mnt/cd1/
cp -R /mnt/cd1/* /srv/repo/6/os/x86_64/
sudo umount /mnt/cd1
sudo mount -t auto -o loop,ro CentOS-6.3-x86_64-bin-DVD2.iso /mnt/cd1/
cp -R /mnt/cd1/Packages/* /srv/repo/6/os/x86_64/Packages/

If you have SELinux enabled, and you should, you’ll need to ensure that Apache’s httpd daemon can read these files with the following:

chcon -Rv --type=httpd_sys_content_t /srv/repo

This will set the SELinux context of the directories and all subfolders to “httpd_sys_content_t”, which is the context httpd can read.
At this point, if your server is named “build” you can do an install from the netinstall CD, and point it to “http://build/centos/6/os/$basearch” as the install point, and all packages will be installed from that location. If you do this “base” install with the 6.4 sources, when 6.5 comes out, read the Changing “base” Warning below.
A point of note, for people who come here confused – I haven’t mentioned “reposync” or “createrepo” tools yet – those will be used much later, or not at all. For the “base” repository, all of the repository metadata is included in the DVD copy.

Setting up the “updates” repository

The updates repository needs to be updated constantly, or it becomes meaningless This is the default channel by which the distribution provides security and bug fixes for the software packages they ship. There are 2 means for synchronizing a remote repository: reposync and rsync. Since rsync is more widely used and easier for many people to understand, and because the benefits of reposync aren’t easy for me to find, we’ll use rsync.

The “updates” repository is large – the 64-bit x86_64 “updates” repository in our lab is currently 14GB, and the “os” repository is 5.6. Each additional architecture and version supported will have similar size requirements.

If you’ve already set up the initial structure from the instructions for the “base” repository, skip this paragraph: First, we need to create our repository structure. This walkthrough will only demo RHEL 6 derivatives, but 5.x will work the same. This is the first time to think about which repositories to build. I’ll create 3 total: “base”, “updates” and “internal”. “base” and “updates” will be the distrobution-provided repositories, and “internal” will be for our company’s internal software. The “base” repository actually lives in a folder called “os”, and each repository has subfolders for each architecture that it supports. Therefore, we need to create 3 directories in /srv/repo/6: “os”, “updates”, and “internal”, and each of those gets 2 subdirectories for our supported architectures: “i386” and “x86_64”. (Also simple with bash expansion.)

mkdir -p /srv/repo/6/{os,updates,internal}/{i386,x86_64}

Now to pull down the repository updates from a mirror. First choose a mirror from the list for your distribution: CentOS, Scientific Linux, or RHEL (ask your support team).
To make sure your “updates” repository doesn’t go out of sync, schedule a cron job, by creating a file in /etc/cron.daily/ or in /etc/cron.hourly/ (See this warning before syncing hourly), called “reposync” and pasting the rsync commands into it (here’s ours):



for vers in 5 6; do

rsync -art rsync://${mirror}/${vers}/os/x86_64 ${repobase}/${vers}/os/
if [ $? -ne 0 ]; then
echo "ERROR getting os files from ${mirror} for ${vers} x64, quitting."
exit 1
rsync -art rsync://${mirror}/${vers}/updates/x86_64 ${repobase}/${vers}/updates/
if [ $? -ne 0 ]; then
echo "ERROR getting updates files from ${mirror} for ${vers} x64, quitting."
exit 1
if [ $vers -eq 5 ]; then
rsync -art rsync://${mirror}/${vers}/os/i386 ${repobase}/${vers}/os/
if [ $? -ne 0 ]; then
echo "ERROR getting os files from ${mirror} for ${vers} i386, quitting."
exit 1
rsync -art rsync://${mirror}/${vers}/updates/i386 ${repobase}/${vers}/updates/
if [ $? -ne 0 ]; then
echo "ERROR getting updates files from ${mirror} for ${vers}, quitting."
exit 1

The first rsync took about 2 hours, but subsequent syncs are much faster, since rsync can skip existing files. You’ll notice 2 things: 1) that we have disabled syncing the 32-bit CentOS 6 updates; 2) that we also rsync the “os” repo – for more on this, read the Changing “base” Warning below. Our lab doesn’t have or test 32-bit CentOS 6, and we always test “latest” releases. Also, I set the particular mirror to be a variable, so that it would be easy to change if required.

Setting clients to use your repositories

The easiest way to set a single client to use your newly-setup repo is to simply edit the /etc/yum.repos.d/*-Base.repo file by hand, but that’s slow. I put a fixed repo file in the root of the webserver (in /repo/yum/, named CentOS-Base.repo. In it, I’ve simply commented out the mirror line, and replaced it with a local baseurl:

name=CentOS-$releasever - Base

#released updates
name=CentOS-$releasever - Updates

This baseurl doesn’t have to be HTTP, since it is a local server and you installed NFS, in that case, use autofs and the file:// URI as:


I haven’t found information that says conclusively if $releasever is properly expanded in that line, but testing shows it works fine in our lab.

Now to test it out on your CentOS 6 client:

wget -O /etc/yum.repos.d/CentOS-Base.repo http://build/CentOS-Base.repo.6
yum clean all
yum upgrade

Now all your yum installs and searches will come from the local repo.

Changing the “base” Source

This is an issue we ran into during testing of this post – you’ll notice all of the rsync scripts pull from “centos/6” not “centos/6.4”, and that we rsync both “updates” and “os” repos nightly.

Which “base” repo you sync against matters. In testing, we initially had the “updates” repo syncing nightly, but not the “os” repo (which was originally staged from a 6.3 DVD. When CentOS 6.4 was released, our “updates” repo shrunk considerably, and we started receiving “Error: Protected multilib versions:” on all updates. This was because “updates” are updates applied against “base”, so they both must be in sync. Therefore, either set up all your systems to sync “6.4” and have no problems, or sync both “base”/”os” and “updates” simultaneously, at the cost of more bandwidth and storage usage.

If you do keep “base” tracking the latest release, your previously-installed systems will continue to upgrade properly – they just will become CentOS 6.4 if they were previously CentOS 6.3. BUT, PXE booting will break. From my colleague:

If you use PXE boot installs, you need to pay attention to when your “base” repo updates so you can copy the appropriate vmlinux and initrd images into the pxe directories.

Staging internally

And a warning about houly syncing.
Most public mirrors will probably blacklist you if you sync hourly. Don’t do that. However, you can set up multiple internal servers, where SERVERA pulls from a public mirror daily, and SERVERB pulls from SERVERA, or even an intermediary. For Example, SERVERA may be the update server your dev lab uses. When patches are verified in dev, you can push updates from SERVERA to a staging location that SERVERB (an SERVERC, etc.) would rsync from, so that production could be “released” to install those patches.

Again: Don’t do hourly syncs to public mirrors.

Series Links

Return to the series header Yum Repositories for Corporate Deployments.

Continue to Part 3 (TBD)


Yum Repositories: Background, Terminology and basics

Part 1 of the series Yum Repositories for Corporate Deployments.

Before we talk about how to set up a new yum repository, we need to discuss why, so that we can set up the right kind of repository. Before we can discuss why to set up a new repository, we need to know what exactly can be set up.



A yum repository is a collection of rpm packages explained by a series of metadata databases. These databases are described in a XML file called “repomd.xml”, and all of this metadata is created automatically by a package appropriately named “createrepo.”


yum is a client (meaning it reaches out to another server) tool to discover software availability from repositories and, if instructed, install that software onto the rpm-compatible GNU/Linux computer. yum uses the rpm database and format to determine dependencies, then handles those dependencies automatically.  For example, if a piece of software requires “python” and python is not installed, yum will find the python package and install it at its latest version, from any of the repositories it has been told of from its repo files.


A repo file is a description to the yum client about where to find a repository, and what to do with the data found therein.  A repo file can describe multiple repositories, that are generally related.  Each repository definition starts with the repository name, and will include a path to the repository (the exported directory on the server which contains the “repodata” metadata directory that describes the repository), the user-friendly name of the repository, and some information about security (gpg signing and protection status).

 Background – Base Repositories

A default Red Hat Enterprise Linux (RHEL, even though Red Hat doesn’t like it) or CentOS or Scientific Linux installation ships with one or several repo files pointing to several different repositories.  Understanding these base repositories helps us decide what we want to set up inside our corporate network.  There are at least 2 default repositories in that configuration, “base”, “updates”.  CentOS and Scientific Linux also ship “extras”, “plus”, and CentOS ships “contrib”.


The “base” repository is the installation copy.  If you’re building a “base” repository in your corporate environment, you simply copy the entire DVD image to the repository location and leave it.  “base” is used to build an initial system to a known state.


Updates is the repository where rpm updates for security, etc. are actually updated.  If you’re building an “updates” repository in your corporate network, you need to sync it from another repository synced to the {RHEL,CentOS,Scientific} original repository.  If you use the correct rsync command, you *only* need to rsync the repository (more in part 2).

extras, plus, and contrib

The best explanations of these repositories for CentOS come from the CentOS team themselves here:

Repository basics

At this point, the repository names are just names.  Only “base” and “updates” have any real meaning – “base” is for the CD install, and “updates” is for security/other updates to “base”.  Any other repository is named so that the client system administrator can understand what might be installed from that repository.

The CentOS Team has some great information, which I’m lifting from :

  • Use of hard-coded version and architecture: ‘baseurl=http: //’ This hard codes both for ‘$releasever’ and ‘$basearch’. Compare this, to the more proper: ‘baseurl=http: //$releasever/en/$basearch/dag’. The ‘hard coded’ approach limits it to only be ‘correct’ for CentOS 4 on an i386 platform.
  • Mixing Fedora repositories with CentOS oriented repositories: Look for ‘name=Fedora’, vs. ‘name=CentOS.(whatever)’. Fedora repositories are not likely to be compatible with CentOS. Repositories for other Enterprise Linux distros derived from the same upstream sources are more likely to be compatible, but should still be used with care.

The same rules will apply to building a corporate yum repository.  For building a corporate repository, I would also add: It’s easier to create more repositories, rather than trying to merge existing ones.  If you are syncing parts of repositories, they should be separate repositories, synced as a single unit.  Yum’s configuration is able to handle many repo files with many repository definitions, as long as those repositories don’t install the same files with conflicting versions.


Continue to Part 2: Setting up a corporate yum repository mirror for bandwidth and staged update management

Return to the series header: Yum Repositories for Corporate Deployments


Yum Repositories for Corporate Deployments

I was tasked with a documentation project regarding use of yum repositories inside a customer’s environment and realized that I was having a hard time finding exactly the information I needed to build the full test environment required to test the documentation.  What will follow is a series of posts covering information I had to research as background.


  1. Background, Terminology and repository basics
  2. Setting up a corporate yum repository mirror for bandwidth and staged update management
  3. Setting up a private repository for additional software


As a quick note up front, based on what I’ve learned, I can’t find a good reason to include the software from a private repository in the update management repository – just build a new repository.

PXE Booting – there are instructions here which will break a PXE Boot environment. Our lab doesn’t PXE or Net boot, so we haven’t tested against them, but a colleague assures me it won’t work.


The following posts were used to build this series. None of them was 100% what I needed, and no source had everything I needed in a single location, but they were all useful, and in no order at all:


Lenovo T430 running Kubuntu 12.10 for extreme battery life.


There’s a new version of this post at, since it’s 4 years newer, you should head there instead.


It’s been a long time since I’ve updated my Linux buildout on this site.

I’ve recently upgraded to a Lenovo T430 (from a T500, and Dell before that). This laptop has the following hardware:

Jump to main sections with these links:
CPU Configuration
Network Configuration
Video/Bumblebee Configuration
Encryption / Security configuration
Battery saving configuration
Custom kernel .config
This is the first time in years I haven’t had a built-in 3G Modem for internet access, but with wifi tethering to my phone, I don’t think it’ll be an issue. Out of the box, everything I cared about worked. I didn’t test the fingerprint reader or the Nvidia graphics with the default install, and pretty quickly customized the system, but if you’re not into customization, rest assured, the Ubuntu team did a fantastic job on the setup.

CPU and Battery

As stated in this post from 2007, I’m a huge fan of extreme battery life. I’m still using cpufreqd, laptop-mode-tools, and checking their configuration with powertop to make sure I’m doing everything I can. I also custom-compile my kjernel, which I’ll discuss more below.

Kubuntu 12.04 defaults to the “ondemand” cpufreq driver, which is great for power savings, except that it does its speed modulation in preference of performance. That is, when the lowest speed of the processor (1.2GHz in my case) isn’t enough, ondemand immediately jumps the CPU speed to the fastest available (2.9GHz in my case). Then, when the fastest is more than required, ondemand steps off slowly. This is perfect for video and gaming applications, and most people. I, however, greatly prefer the “conservative” driver, which works in the opposite manner: when an application needs more power, the conservative driver steps up 1 CPU speed level at a time, until the appropriate CPU speed is reached. Then, when the utilization drops off, conservative immediately drops all the way to the slowest speed, to step back up again if needed.
Using cpufreqd allows me to control this even more granularily, while not getting in the way of the kernel modules. My configuration uses “ondemand” when I’m plugged in, and “conservative” when I’m on battery power. The small delay in performance is worth the added minutes in battery life, especially since most of my on-battery time is very low-demand applications.
Since sometime in 2011, Ubuntu, however, has not shipped a working cpufreqd daemon – it’s apparently broken in the upstream Debian as well, and is well documented in this Launchpad bug. So I downloaded the cpufreqd-dev source package, the patch, and rebuilt cpufreqd myself. Now that it’s working, I can use the attached updated cpufreqd.conf configuration.


Both the ethernet and wireless adapters work right on first install with Kubuntu 12.10.  The wireless uses the iwlwifi driver, and connects to my router at 104mbps.  I’m still using the script from this post to get my MTU set to 9000, rather than 1500, for jumbo frames support when in the main office network. This has a major effect on network speed for large transfers, but most networks don’t support it.h

The Wireless adapter needed no changes, and NetworkManager handles it beautifully, even when tethering to my phone.


I haven’t had any sound issues, but a few users have reported problems when using the docking station. According to this post on ThinkPad Forums, the solution is to simply edit /etc/modprobe.d/alsa-base.conf and add:
options snd-hda-intel model=thinkpad
This causes no issues on my system.


This laptop ships with two video cards, an Intel (which uses the kernel i915 driver) low-power adapter, and an Nvideo high-performance, high-power adapter. In Windows, you can click an application to switch between the two for all applications. The Nvidia adapter uses about 10W more power than the Intel card, which means using the Nvidia adapter alone halves the system’s battery life.
By default, the Ubuntu kernel enables a feature called “vgaswitcheroo” which is well documented on the Ubuntu help site. I had a hard time getting it to work with my custom kernel, though, even though it was enabled. KDE and lightdm just didn’t want to swtich the laptop panel over to the Nvidia card. This *may* have something to do with my BIOS settings, which I currently can’t change due to office IT restrictions.
There is a new project called Bumblebee, which allows the user to use the Intel card, and only turn the Nvidia on for some applications. This gives the best of both worlds for power and battery savings, but is a work-in-progress, and not all applications run under Bumblebee. I’m using the Primus additions. Installation of Bumblebee is documented here, and Primus installation is documented here. I didn’t have to make any changes to get these installs working with simple applications.

VMware Workstation 9

VMware Workstation 9, however, offered some interesting challenges with Bumblebee. Out of the box, Vmware Workstation 9 installed (even with my custom kernel), and ran great, but would always give a warning that 3D acceleration was not available, which I expected while using the Intel card. However, Bumblebee has some limitations which mean it can’t run VMware workstation by default. I wrote a script to handle this, which I wrote up last night. I’m using a lot of work of others, so follow the links on that page to cmillersp’s post on the VMware Community Forums.

Encryption and Security

During installation, I chose the option to use an encrypted LVM volume. This uses DM-Crypt to encrypt the full HDD, so that it has to be unlocked at boot time. The Kubuntu installer seems to forget this fact, so it also asks you to set up ecryptfs private home directories, which is NOT neccessary for a single-user laptop, since the whole OS is already encrypted. The only oddity with dm-crypt is that sometimes the splash screen prompt to unlock the computer doesn’t show. If I just wait for disk activity to disappear, and have a blank screen, I can just type the passphrase, and it’ll still unlock successfully.

I don’t have the fingerprint reader set up, but if I do, I’ll update this post.

Battery and Power Savings

First, I use the configuration in the CPU Configuration section above for cpufreqd. Then I use laptop-mode-tools to set other configuration settings. I’ve attached all the files I have modified here, and it’s fairly power-saving aggressive. The only thing I should do, but don’t, is to disable the bluetooth adapter when I’m on battery, since it uses 1W just for the adapter. However, I have bluetooth headphones and a bluetooth mouse, which is why I have the bluetooth adapter in the laptop to begin with, so disabling it removes some critical functionality. I DO have it set to autosuspend, which is a little annoying when I go back to the mouse after 5 seconds of inactivity, but the annoyance is worth the savings, especially when I’m writing a long post like this.
All of these go in /etc/laptop-mode/conf.d :

With these settings, I was able to write this whole post, with the bluetooth mouse connected, running at an average of 13.2W. In 3.4 hours (I was doing other tasks, including feeding and changing my napping baby), I used 41% of my 93.6Wh battery. If I were to take this on a plane, I’d kill the wireless and bluetooth for probably another 2W savings, but I’d do that by hand.

Kernel Configuration

I have been building a custom kernel for my laptop for about 6 years now. The default Ubuntu image uses “generic-x86_64” for the processor family, but all of my laptops are “Core2 / Newer Xeon”. Just making that single change to my kernel results in about 0.5W-1W less power consumption, due to the increased efficiency gained by the kernel knowing about the new processor registers and commands that aren’t available to older processors. This greater CPU efficiency also means lower temperatures, and therefore lower fan speed.

My kernel configuration is attached here. Build it by following the instructions at the Ubuntu Help site

sudo apt-get install linux-kernel-devel fakeroot kernel-wedge build-essential
sudo apt-get install linux-source
sudo usermod -a -G src YOUR_USERNAME

Now log out and back in, so that you’re a member of the “src” group.

cd /usr/src
sudo chown -R $USER:src .
tar -jxf ./linux-source-3.5.0/linux-source-3.5.0.tar.bz2
ln -s linux-source-3.5.0 linux
cd linux
mv rob-config-20121204c.txt .config
make oldconfig
make menuconfig

Make any changes you want in here, then exit and save

fakeroot make-kpkg --initrd --append-to-version=.20121204c kernel_image kernel_headers

You’ll get 2 DEB files in /usr/src that you can then install and boot to. the “append-to-version” I use as a dating system for my kernels. “20121204c” means the 4th kernel attempt on December 4th, 2012, the day I recieved this laptop.

All Modified Files

Laptop-mode-tools config:
All of these go in /etc/laptop-mode/conf.d :
Other configs:
custom kernel .config.


Bumblebee / Primus and VMware Workstation (nvidia optimus graphics on Ubuntu)

I have a new Lenovo T430 with Nvidia/Intel hybrid graphics. The Intel card is for power saving, the Nvidia card for performance. The easiest way to handle this setup is to just choose the correct card on bootup, but this is inconvenient. Bumblebee provides a way to use the Intel card most of the time, but use the Nvidia for high-performance tasks such as games, and ONLY for those apps. Once it’s set up, it works pretty well, except for a few apps that require additional tweaking, such as Steam, or Wine.

One of the apps that requires tweaking is VMware Workstation (I’m running WKS 9). cmillersp provided a great write-up on the VMware Communities, which is what I set up on my system. VMware Workstation runs great – I can load OpenGL 3D apps in VMs and everything runs fantastic.

Except that I run VMs all the time, and using the Nvidia card all the time kills the performance benefit of the Intel card twice: once because I’m using the Nvidia card, and once again because BOTH GPUs are running at the same time. So I wanted a way to dynamically choose which card to run VMware under, based on whether I was on AC power or battery power.

The result is the attached script below, which I’ll be submitting to the Bumblebee wiki / project, as well as the VMware forums. This is version 1, which works as follows:

  1. Must be installed by “sudo ~/bin/vmware –install” – it will make the cd /usr/lib/vmware/bin; mv vmware-vmx vmware-vmx.real and then do the script creation at /usr/lib/vmware/bin/vmware-vmx, so that it’s a single portable script.
  2. Takes several options to force using the Nvidia card or not
  3. If no force option is applied, automatically determines AC adapter state or battery charge/discharge state
  4. Based on the above, decides whether to launch /usr/bin/vmware normally, or via the instructions from cmillersp

I’ve done some additional work to make it run via either “primusrun” or “optirun”. Optirun is much slower, but at least functional with fewer installs. I hope this is useful to someone else!

Edit: 2012-12-17 – v1.1 added gksu auto-detection
Edit: 2014-01-08 – Moved to github repo:

About Us

Complete networking solutions for business.
September 2017
« Aug