[1617] | 1 | Upgrading Scripts for a new Fedora distribution |
---|
| 2 | =============================================== |
---|
| 3 | |
---|
| 4 | 1. Gather knowledge |
---|
| 5 | ------------------- |
---|
| 6 | |
---|
| 7 | You should read the Release Notes for all of the intervening |
---|
| 8 | releases. For example, here are the Fedora 13 release notes: |
---|
| 9 | |
---|
| 10 | http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/ |
---|
| 11 | |
---|
| 12 | Because we sometimes skip releases, you should read any skipped |
---|
| 13 | release's report notes. |
---|
| 14 | |
---|
| 15 | Example: |
---|
| 16 | |
---|
| 17 | In Fedora 12, i586 was deprecated in favor of i686; this meant |
---|
| 18 | that any parts of Scripts that referenced i586 explicitly had to |
---|
| 19 | changed to i686. |
---|
| 20 | |
---|
| 21 | 2. Update the Scripts build environment |
---|
| 22 | --------------------------------------- |
---|
| 23 | |
---|
| 24 | A large amount of the Scripts source repository is Fedora Release |
---|
| 25 | specific, so when you are ramping up the new release, you will want |
---|
| 26 | a new branch to do development on, before merging back upon the |
---|
| 27 | official release. You can do this with: |
---|
| 28 | |
---|
| 29 | svn cp svn://scripts.mit.edu/trunk \ |
---|
| 30 | svn://scripts.mit.edu/branches/fcXX-dev |
---|
| 31 | |
---|
| 32 | On the new branch, there are a number of files you will have to |
---|
| 33 | update: |
---|
| 34 | |
---|
| 35 | 2.1 Mock |
---|
| 36 | |
---|
| 37 | Mock needs to be setup for the new environment. The first thing to do |
---|
| 38 | is to update the Makefile by substituting |
---|
| 39 | s/scripts-fcOLD/scripts-fcNEW/g on the /usr/bin/mock invocations. |
---|
| 40 | After that, you need to go to /etc/mock and create the new cfg file |
---|
| 41 | for the new scripts-fcXX-ARCH configurations (where ARCH is x86_64 and |
---|
| 42 | i386). You can base the new cfg off of the older version's, however |
---|
| 43 | you will want to make the following changes: |
---|
| 44 | |
---|
| 45 | * Update all references to the old Fedora release to the new |
---|
| 46 | Fedora release. This includes root, dist, mirrorlist, baseurl |
---|
| 47 | |
---|
| 48 | * Temporarily disabling the web.mit.edu Scripts RPM repository |
---|
| 49 | and the local RPM repository by setting enabled=0 (it's there for |
---|
| 50 | a reason!) However, the local RPM repository is fairly painless |
---|
| 51 | to create and will come in handy when you start attempting to |
---|
| 52 | build packages that have dependencies on other scriptsified |
---|
| 53 | packages: you can set one up as scripts-build with: |
---|
| 54 | |
---|
| 55 | mkdir ~/mock-local |
---|
| 56 | createrepo ~/mock-local |
---|
| 57 | |
---|
| 58 | 3. Rebuild Scripts packages |
---|
| 59 | --------------------------- |
---|
| 60 | |
---|
| 61 | In order to support specific extra functionality, we have scriptsified |
---|
| 62 | a variety of Fedora packages. When the base packages get upgrades, |
---|
| 63 | we need to upgrade the scriptsification. Some of the following topics |
---|
| 64 | are covered in 'package-build-howto', but a new Fedora release tends |
---|
| 65 | to also result in somewhat rarer situations. |
---|
| 66 | |
---|
[1629] | 67 | As you finish building packages, you'll want to place them somewhere |
---|
| 68 | so they don't get blown away on a successive mock build. ~/mock-local |
---|
| 69 | is a good choice. The Mock RPMs will be created in: |
---|
| 70 | |
---|
| 71 | /var/lib/mock/$MOCK_ENV/result/ |
---|
| 72 | |
---|
[1617] | 73 | Here are some of the common troubles you'll have to deal with: |
---|
| 74 | |
---|
| 75 | 3.1 Spec patches are no longer necessary |
---|
| 76 | |
---|
| 77 | When a Fedora release gets EOL'ed, we may continue to backport |
---|
| 78 | patches for CVE's manually. When we upgrade to a non-EOL'd release, |
---|
| 79 | those patches will generally become unnecessary and can be dropped. |
---|
| 80 | |
---|
| 81 | You can drop a modified specfile from the repository simply by |
---|
| 82 | `svn rm`ing: |
---|
| 83 | |
---|
| 84 | * The spec patch in server/fedora/specs, |
---|
| 85 | * The source code patch in server/common/patches, and |
---|
| 86 | * The upstream_yum entry in server/fedora/Makefile |
---|
| 87 | |
---|
| 88 | If a specfile merely bumps the version field, there may be no extra |
---|
| 89 | patch (this indicates that the maintainer rebuilt the package simply |
---|
| 90 | by manually dropping the new source tarball in rpmbuild/SOURCES, |
---|
| 91 | which is kind of sketchy but works. See -c 1586 for an example.) |
---|
| 92 | |
---|
| 93 | 3.2 Spec patches no longer apply |
---|
| 94 | |
---|
| 95 | Symptom: |
---|
| 96 | |
---|
| 97 | $ make patch-specs |
---|
| 98 | patching file openssh.spec |
---|
| 99 | Hunk #1 succeeded at 74 with fuzz 2 (offset 11 lines). |
---|
| 100 | Hunk #2 failed at 88. |
---|
| 101 | Hunk #3 succeeded at 177 (offset 14 lines). |
---|
| 102 | Hunk #4 succeeded at 270 with fuzz 2 (offset 36 lines). |
---|
| 103 | 1 out of 4 hunks failed--saving rejects to openssh.spec.rej |
---|
| 104 | |
---|
| 105 | Fix: |
---|
| 106 | |
---|
| 107 | The main thing to remember is where the generated files live |
---|
| 108 | they are placed in rpmbuild/SPECS/openssh.spec{.rej,.orig}. |
---|
| 109 | A workflow for fixing them might look like: |
---|
| 110 | |
---|
| 111 | 1. Inspect the rejects file. |
---|
| 112 | 2. As much as possible, manually fix the original diff |
---|
| 113 | file in /srv/repository/server/fedora/specs |
---|
| 114 | 3. If absolutely necessary, edit the rpmbuild/SPECS/openssh.spec |
---|
| 115 | file with any final changes (this is dangerous because |
---|
| 116 | this file is blown away on a successive make) |
---|
| 117 | 4. Generate a new unified diff: |
---|
| 118 | diff -u openssh.spec.orig openssh.spec > \ |
---|
| 119 | /srv/repository/server/fedora/specs\openssh.spec.patch |
---|
| 120 | |
---|
| 121 | 3.3 Mock fails with no error message |
---|
| 122 | |
---|
| 123 | Fix: You forgot to add scripts-build to the mock group. See |
---|
| 124 | https://bugzilla.redhat.com/show_bug.cgi?id=630791 |
---|
| 125 | [XXX: remove this entry when this bug is fixed] |
---|
| 126 | |
---|
| 127 | 3.4 Source patches no longer apply |
---|
| 128 | |
---|
| 129 | Symptoms: |
---|
| 130 | |
---|
| 131 | Generally, you will see these error messages after Mock starts |
---|
| 132 | building (if they occur before Mock, that means it's a bug in the |
---|
| 133 | spec patch, not a source patch that the spec patch references.) |
---|
| 134 | |
---|
| 135 | Fix: |
---|
| 136 | |
---|
| 137 | The error message will be from within a schroot that Mock is using. |
---|
| 138 | As a result, it's not immediately obvious where the files live. |
---|
| 139 | |
---|
[1629] | 140 | The easiest approach is to use rpmbuild to manually reapply the |
---|
| 141 | patches. |
---|
[1617] | 142 | |
---|
[1629] | 143 | rpmbuild -bp path/to/foo.spec |
---|
[1617] | 144 | |
---|
[1629] | 145 | If this fails complaining about a dependency, you should install |
---|
| 146 | the dependency and add it to the Makefile. |
---|
| 147 | |
---|
| 148 | Once you've fixed the patch, you can rerun rpmbuild after running |
---|
| 149 | |
---|
| 150 | make setup |
---|
| 151 | |
---|
| 152 | (This is useful if you can't do a full make due to another mock |
---|
| 153 | process running.) |
---|
| 154 | |
---|
[1617] | 155 | 4. "Officializing" everything |
---|
| 156 | ----------------------------- |
---|
| 157 | |
---|
| 158 | web.mit.edu scripts repository (/mit/scripts/rpm-fcXX and |
---|
[1644] | 159 | /mit/scripts/rpm-fcXX-testing) needs to be made. It's quite simple; |
---|
| 160 | all you need to do is copy the RPMs from the build server to there |
---|
[2066] | 161 | (probably going through a trusted machine, since you don't want to put |
---|
| 162 | your root tickets on a server.) When you're done, run `createrepo -d` |
---|
[1644] | 163 | on the directory. |
---|
[1617] | 164 | |
---|
[1644] | 165 | Note that if you do a successive rebuild without bumping the Subversion |
---|
| 166 | revision (via a `svn up`), the new package will have the *same* version |
---|
| 167 | and yum will probably insist on using the old cached version. You can |
---|
| 168 | use `yum clean all` to reset your cache and force yum to get the latest |
---|
| 169 | version. |
---|
| 170 | |
---|
[1661] | 171 | 5. Update fs sysname |
---|
| 172 | -------------------- |
---|
| 173 | |
---|
[2246] | 174 | XXX out of date |
---|
[1661] | 175 | Update /etc/sysconfig/openafs with an extra amd64_fedoraX_scripts and |
---|
| 176 | amd64_fedoraX sysname. The format should be evident from the existing |
---|
| 177 | entries. [XXX There might be other things you want] |
---|
| 178 | |
---|
[2246] | 179 | 6. Bind to scripts-test |
---|
| 180 | ----------------------- |
---|
| 181 | |
---|
| 182 | First, make sure no other servers are bound to scripts-test (try ping). |
---|
| 183 | Then, create /etc/sysconfig/network-scripts/ifcfg-eth0:0 based off of |
---|
| 184 | /etc/sysconfig/network-scripts/ifcfg-eth0 but with the scripts-test |
---|
| 185 | IP address 18.181.0.229. Run `ifup eth0:0` to complete the change. |
---|
| 186 | |
---|
| 187 | 7. Testing critical infrastructure |
---|
[1695] | 188 | ---------------------------------- |
---|
| 189 | |
---|
| 190 | There are some important Scripts maintained applications you should test |
---|
| 191 | and ensure run on the new platform. They include: |
---|
| 192 | |
---|
| 193 | - http://scripts.mit.edu |
---|
| 194 | - http://scripts.mit.edu/wiki |
---|
| 195 | - http://scripts.mit.edu/trac |
---|
| 196 | - http://scripts.mit.edu/whois/ |
---|
| 197 | - http://pony.scripts.mit.edu |
---|
| 198 | |
---|
[2246] | 199 | 8. Extra stuff |
---|
[1617] | 200 | -------------- |
---|
| 201 | |
---|
| 202 | Fedora occasionally updates the architecture name for 32-bit; the last |
---|
| 203 | such update was in Fedora 12, when i586 became i686. Fixing this |
---|
| 204 | usually just involves replacing i586 with i686 in the appropriate places |
---|
| 205 | (Makefile, specfiles, /etc/mock configuration). Note that for |
---|
| 206 | hysterical raisins we still refer to our 32-bit builds as i386. |
---|
| 207 | [XXX: Maybe this should change] |
---|
[2066] | 208 | |
---|
| 209 | Until we decide that the performance impact is negligible, any new PHP |
---|
| 210 | extensions other than the few we’ve whitelisted should be disabled by |
---|
| 211 | emptying their .ini files in /etc/php.d. |
---|
| 212 | |
---|
[2246] | 213 | 9. Sending announcements |
---|
[2066] | 214 | ------------------------ |
---|
| 215 | |
---|
| 216 | Once development work has finished, we need to allow users to test |
---|
| 217 | their websites on the new servers. |
---|
| 218 | |
---|
| 219 | SIPB Internal Testing: Send an email to scripts-team@mit.edu |
---|
| 220 | and -c sipb notifying them of testing procedure and known |
---|
| 221 | issues. |
---|
| 222 | |
---|
| 223 | General Testing: |
---|