1 | Upgrading Scripts for a new Fedora distribution |
---|
2 | =============================================== |
---|
3 | |
---|
4 | 1. Gather knowledge |
---|
5 | ------------------- |
---|
6 | |
---|
7 | You should read the Release Notes for all of the intervening |
---|
8 | releases. For example, here are the Fedora 13 release notes: |
---|
9 | |
---|
10 | http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/ |
---|
11 | |
---|
12 | Because we sometimes skip releases, you should read any skipped |
---|
13 | release's report notes. |
---|
14 | |
---|
15 | Example: |
---|
16 | |
---|
17 | In Fedora 12, i586 was deprecated in favor of i686; this meant |
---|
18 | that any parts of Scripts that referenced i586 explicitly had to |
---|
19 | changed to i686. |
---|
20 | |
---|
21 | 2. Update the Scripts build environment |
---|
22 | --------------------------------------- |
---|
23 | |
---|
24 | A large amount of the Scripts source repository is Fedora Release |
---|
25 | specific, so when you are ramping up the new release, you will want |
---|
26 | a new branch to do development on, before merging back upon the |
---|
27 | official release. You can do this with: |
---|
28 | |
---|
29 | svn cp svn://scripts.mit.edu/trunk \ |
---|
30 | svn://scripts.mit.edu/branches/fcXX-dev |
---|
31 | |
---|
32 | On the new branch, there are a number of files you will have to |
---|
33 | update: |
---|
34 | |
---|
35 | 2.1 Mock |
---|
36 | |
---|
37 | Mock needs to be setup for the new environment. The first thing to do |
---|
38 | is to update the Makefile by substituting |
---|
39 | s/scripts-fcOLD/scripts-fcNEW/g on the /usr/bin/mock invocations. |
---|
40 | After that, you need to go to /etc/mock and create the new cfg file |
---|
41 | for the new scripts-fcXX-ARCH configurations (where ARCH is x86_64 and |
---|
42 | i386). You can base the new cfg off of the older version's, however |
---|
43 | you will want to make the following changes: |
---|
44 | |
---|
45 | * Update all references to the old Fedora release to the new |
---|
46 | Fedora release. This includes root, dist, mirrorlist, baseurl |
---|
47 | |
---|
48 | * Temporarily disabling the web.mit.edu Scripts RPM repository |
---|
49 | and the local RPM repository by setting enabled=0 (it's there for |
---|
50 | a reason!) However, the local RPM repository is fairly painless |
---|
51 | to create and will come in handy when you start attempting to |
---|
52 | build packages that have dependencies on other scriptsified |
---|
53 | packages: you can set one up as scripts-build with: |
---|
54 | |
---|
55 | mkdir ~/mock-local |
---|
56 | createrepo ~/mock-local |
---|
57 | |
---|
58 | 3. Rebuild Scripts packages |
---|
59 | --------------------------- |
---|
60 | |
---|
61 | In order to support specific extra functionality, we have scriptsified |
---|
62 | a variety of Fedora packages. When the base packages get upgrades, |
---|
63 | we need to upgrade the scriptsification. Some of the following topics |
---|
64 | are covered in 'package-build-howto', but a new Fedora release tends |
---|
65 | to also result in somewhat rarer situations. |
---|
66 | |
---|
67 | As you finish building packages, you'll want to place them somewhere |
---|
68 | so they don't get blown away on a successive mock build. ~/mock-local |
---|
69 | is a good choice. The Mock RPMs will be created in: |
---|
70 | |
---|
71 | /var/lib/mock/$MOCK_ENV/result/ |
---|
72 | |
---|
73 | Here are some of the common troubles you'll have to deal with: |
---|
74 | |
---|
75 | 3.1 Spec patches are no longer necessary |
---|
76 | |
---|
77 | When a Fedora release gets EOL'ed, we may continue to backport |
---|
78 | patches for CVE's manually. When we upgrade to a non-EOL'd release, |
---|
79 | those patches will generally become unnecessary and can be dropped. |
---|
80 | |
---|
81 | You can drop a modified specfile from the repository simply by |
---|
82 | `svn rm`ing: |
---|
83 | |
---|
84 | * The spec patch in server/fedora/specs, |
---|
85 | * The source code patch in server/common/patches, and |
---|
86 | * The upstream_yum entry in server/fedora/Makefile |
---|
87 | |
---|
88 | If a specfile merely bumps the version field, there may be no extra |
---|
89 | patch (this indicates that the maintainer rebuilt the package simply |
---|
90 | by manually dropping the new source tarball in rpmbuild/SOURCES, |
---|
91 | which is kind of sketchy but works. See -c 1586 for an example.) |
---|
92 | |
---|
93 | 3.2 Spec patches no longer apply |
---|
94 | |
---|
95 | Symptom: |
---|
96 | |
---|
97 | $ make patch-specs |
---|
98 | patching file openssh.spec |
---|
99 | Hunk #1 succeeded at 74 with fuzz 2 (offset 11 lines). |
---|
100 | Hunk #2 failed at 88. |
---|
101 | Hunk #3 succeeded at 177 (offset 14 lines). |
---|
102 | Hunk #4 succeeded at 270 with fuzz 2 (offset 36 lines). |
---|
103 | 1 out of 4 hunks failed--saving rejects to openssh.spec.rej |
---|
104 | |
---|
105 | Fix: |
---|
106 | |
---|
107 | The main thing to remember is where the generated files live |
---|
108 | they are placed in rpmbuild/SPECS/openssh.spec{.rej,.orig}. |
---|
109 | A workflow for fixing them might look like: |
---|
110 | |
---|
111 | 1. Inspect the rejects file. |
---|
112 | 2. As much as possible, manually fix the original diff |
---|
113 | file in /srv/repository/server/fedora/specs |
---|
114 | 3. If absolutely necessary, edit the rpmbuild/SPECS/openssh.spec |
---|
115 | file with any final changes (this is dangerous because |
---|
116 | this file is blown away on a successive make) |
---|
117 | 4. Generate a new unified diff: |
---|
118 | diff -u openssh.spec.orig openssh.spec > \ |
---|
119 | /srv/repository/server/fedora/specs\openssh.spec.patch |
---|
120 | |
---|
121 | 3.3 Mock fails with no error message |
---|
122 | |
---|
123 | Fix: You forgot to add scripts-build to the mock group. See |
---|
124 | https://bugzilla.redhat.com/show_bug.cgi?id=630791 |
---|
125 | [XXX: remove this entry when this bug is fixed] |
---|
126 | |
---|
127 | 3.4 Source patches no longer apply |
---|
128 | |
---|
129 | Symptoms: |
---|
130 | |
---|
131 | Generally, you will see these error messages after Mock starts |
---|
132 | building (if they occur before Mock, that means it's a bug in the |
---|
133 | spec patch, not a source patch that the spec patch references.) |
---|
134 | |
---|
135 | Fix: |
---|
136 | |
---|
137 | The error message will be from within a schroot that Mock is using. |
---|
138 | As a result, it's not immediately obvious where the files live. |
---|
139 | |
---|
140 | The easiest approach is to use rpmbuild to manually reapply the |
---|
141 | patches. |
---|
142 | |
---|
143 | rpmbuild -bp path/to/foo.spec |
---|
144 | |
---|
145 | If this fails complaining about a dependency, you should install |
---|
146 | the dependency and add it to the Makefile. |
---|
147 | |
---|
148 | Once you've fixed the patch, you can rerun rpmbuild after running |
---|
149 | |
---|
150 | make setup |
---|
151 | |
---|
152 | (This is useful if you can't do a full make due to another mock |
---|
153 | process running.) |
---|
154 | |
---|
155 | 4. "Officializing" everything |
---|
156 | ----------------------------- |
---|
157 | |
---|
158 | web.mit.edu scripts repository (/mit/scripts/rpm-fcXX and |
---|
159 | /mit/scripts/rpm-fcXX-testing) needs to be made. It's quite simple; |
---|
160 | all you need to do is copy the RPMs from the build server to there |
---|
161 | (probably going through a trusted machine, since you don't want to put |
---|
162 | your root tickets on a server.) When you're done, run `createrepo -d` |
---|
163 | on the directory. |
---|
164 | |
---|
165 | Note that if you do a successive rebuild without bumping the Subversion |
---|
166 | revision (via a `svn up`), the new package will have the *same* version |
---|
167 | and yum will probably insist on using the old cached version. You can |
---|
168 | use `yum clean all` to reset your cache and force yum to get the latest |
---|
169 | version. |
---|
170 | |
---|
171 | 5. Update fs sysname |
---|
172 | -------------------- |
---|
173 | |
---|
174 | Update /etc/sysconfig/openafs with an extra amd64_fedoraX_scripts and |
---|
175 | amd64_fedoraX sysname. The format should be evident from the existing |
---|
176 | entries. [XXX There might be other things you want] |
---|
177 | |
---|
178 | 6. Testing critical infrastructure |
---|
179 | ---------------------------------- |
---|
180 | |
---|
181 | There are some important Scripts maintained applications you should test |
---|
182 | and ensure run on the new platform. They include: |
---|
183 | |
---|
184 | - http://scripts.mit.edu |
---|
185 | - http://scripts.mit.edu/wiki |
---|
186 | - http://scripts.mit.edu/trac |
---|
187 | - http://scripts.mit.edu/whois/ |
---|
188 | - http://pony.scripts.mit.edu |
---|
189 | |
---|
190 | 7. Extra stuff |
---|
191 | -------------- |
---|
192 | |
---|
193 | Fedora occasionally updates the architecture name for 32-bit; the last |
---|
194 | such update was in Fedora 12, when i586 became i686. Fixing this |
---|
195 | usually just involves replacing i586 with i686 in the appropriate places |
---|
196 | (Makefile, specfiles, /etc/mock configuration). Note that for |
---|
197 | hysterical raisins we still refer to our 32-bit builds as i386. |
---|
198 | [XXX: Maybe this should change] |
---|
199 | |
---|
200 | Until we decide that the performance impact is negligible, any new PHP |
---|
201 | extensions other than the few we’ve whitelisted should be disabled by |
---|
202 | emptying their .ini files in /etc/php.d. |
---|
203 | |
---|
204 | 8. Sending announcements |
---|
205 | ------------------------ |
---|
206 | |
---|
207 | Once development work has finished, we need to allow users to test |
---|
208 | their websites on the new servers. |
---|
209 | |
---|
210 | SIPB Internal Testing: Send an email to scripts-team@mit.edu |
---|
211 | and -c sipb notifying them of testing procedure and known |
---|
212 | issues. |
---|
213 | |
---|
214 | General Testing: |
---|