Files listed twice question

[Adrián Márques]

Hi there,

I'm an absolute newbie trying to package an application with RPM for the
first time and would like to mark some files as config files.
The top directory of this aplication has several files and directories,
only two of which hold config files. Thus, I'm doing something like this
in my spec file:

%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*

Obviously, this generates several 'file listed twice' warnings. However,
I queried the generated rpm and didn't find anything wrong with it. All
config files where included and correctly marked as config files.

So my questions are: can I safely ignore these warnings or listing files
twice like I have causes a problem I'm not seeing now? is there a better
way to do what I want?

I have been looking through the list archives so I know many of you
would advice me to explicitly list all files. I know this would take
care of this particular problem, but I don't want to do that unless I
really have to, since I find globbing significantly more practical in
this case.

I thank you already for taking the time to read this.

Regards.

Adrián.

[Valery Reznic]

Subject: Files listed twice question
Date: Wednesday, September 24, 2008, 9:28 PM
Hi there,
I'm an absolute newbie trying to package an application
with RPM for the
first time and would like to mark some files as config
files.
The top directory of this aplication has several files and
directories,
only two of which hold config files. Thus, I'm doing
something like this
%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed
twice' warnings. However,
I queried the generated rpm and didn't find anything
wrong with it. All
config files where included and correctly marked as config
files.
So my questions are: can I safely ignore these warnings or
listing files
twice like I have causes a problem I'm not seeing now?
is there a better
way to do what I want?
I have been looking through the list archives so I know
many of you
would advice me to explicitly list all files. I know this
would take
care of this particular problem, but I don't want to do
that unless I
really have to, since I find globbing significantly more
practical in
this case.

It's strange that it's only warning at least before it was error.

I personally prefer am list file explicit. It'sallows cross-check between installation files into build-root and filelist.
If you haven't too much files I suggest explicitly list them. If you have too much you can use find to generate filelist on the fly.

Valery

I thank you already for taking the time to read this.
Regards.
Adrián.
_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

[Adrián Márques]

Firstly, thank you for your answer Valery.

I didn't think of including this in my previous mail, but your comments
tells me it's relevant: I'm using rpm version 4.4.2 on CentOS 5.2.

Did I mention I'm the epitome of laziness? I've read about using find to
auto generate the file list, but then I'd have to 1) make the script and
2) run it for every release prior to packaging. Whereas the individual
files are bound to change significantly between releases, the config
dirs are pretty much fixed, so a setup like the one I had in mind would
be a one-time deal which I could unscrupulously copy/paste for all
future releases, thus taking me a step closer towards doing absolutely
nothing in exchange for my salary, which is my ultimate goal in life.

Post by Valery Reznic

It allows cross-check between installation files into build-root and

filelist

you mean there's those extra checks such as that if I list a file that
is missing in the build-root the build will fail, and if the opposite
happens I'll be warned that there are files I haven't listed in my
build-root?
If so, I can't see the benefit, since (as I understood it) I'd be
running the find script to list whatever files end up in my build root
anyway, so in effect, there'd be no difference to globbing regarding
what files I'm including. If possible, please clarify this a bit for me
in the very likely case that I'm missing something.

As I gathered from past discussions the good thing about listing files
explicitly was that you could easily compare spec files to find out what
changed between releases. Are there other advantages?

All that being said, my original questions still stand. Can the warnings
be ignored? Is there a better way of doing what I want without
explicitly listing files?

Thank you all.

Adrián.

Post by Valery Reznic

Subject: Files listed twice question
Date: Wednesday, September 24, 2008, 9:28 PM
Hi there,
I'm an absolute newbie trying to package an application
with RPM for the
first time and would like to mark some files as config
files.
The top directory of this aplication has several files and
directories,
only two of which hold config files. Thus, I'm doing
something like this
%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed
twice' warnings. However,
I queried the generated rpm and didn't find anything
wrong with it. All
config files where included and correctly marked as config
files.
So my questions are: can I safely ignore these warnings or
listing files
twice like I have causes a problem I'm not seeing now?
is there a better
way to do what I want?
I have been looking through the list archives so I know
many of you
would advice me to explicitly list all files. I know this
would take
care of this particular problem, but I don't want to do
that unless I
really have to, since I find globbing significantly more
practical in
this case.

It's strange that it's only warning at least before it was error.
I personally prefer am list file explicit. It'sallows cross-check between installation files into build-root and filelist.
If you haven't too much files I suggest explicitly list them. If you have too much you can use find to generate filelist on the fly.
Valery

I thank you already for taking the time to read this.
Regards.
Adrián.

[G***@aotx.uscourts.gov]

Post by Adrián Márques
I'm an absolute newbie trying to package an application with RPM for the
first time and would like to mark some files as config files.
The top directory of this aplication has several files and directories,
only two of which hold config files. Thus, I'm doing something like this
%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed twice' warnings. However,
I queried the generated rpm and didn't find anything wrong with it. All
config files where included and correctly marked as config files.
So my questions are: can I safely ignore these warnings or listing files
twice like I have causes a problem I'm not seeing now? is there a better
way to do what I want?

You could ignore them, but it is not recommended. The problem is that you
are effectively listing them twice the way you have it above. Your first
entry "/usr/local/myAppTopDir/" tells it to add that directory and
everything under it. Your next two entries and inside that first entry's
list.

Post by Adrián Márques
I have been looking through the list archives so I know many of you
would advice me to explicitly list all files. I know this would take
care of this particular problem, but I don't want to do that unless I
really have to, since I find globbing significantly more practical in
this case.

I agree that globbing tends to be a bit easier to handle in some cases as
you are learning. You could list your first entry with a %doc prefix, and
then provide more specific blobbing for the rest of that first entries
contents.

Does that make sense?


Another method you can try, and based on your later comments I think you've
realized the debate-ability of auto-listing, is something like what will
follow. I do this to gather up file lists for when I package Informix:

%define INSTALLDIR /opt/informix

%install
<snip>
find $INSTALLDIR -printf "%y&attr(%m,%u,%g) %p\n" | grep -v
"$INSTALLDIR\/etc\/.*" | sed 's/^d/\%dir /' | sed 's/&/%/' | sed
's/^[^%]//' | sed 's!%{buildroot}!!' > %{_tmppath}/%{name}-files
find $INSTALLDIR/etc -printf "%y&config &attr(%m,%u,%g) %p\n" -not -type d
| grep -v "^[dl]" |grep -v "$INSTALLDIR\/etc$"| sed 's/&/%/g' | sed
's/^f//' | sed 's!%{buildroot}!!' >> %{_tmppath}/%{name}-files

%files -f %{_tmppath}/%{name}-files


Now... its a tad convoluted, but basically the first find does this:
1: find everything in the $INSTALLDIR and print it to stdout as
"<type>&attr(<mode>,<uid>,<gid>) /path/to/file\n"
2: Ignore all files in my $INSTALLDIR/etc directory cause I want to specify
them later as config files
3: If a line starts with d its a directory, so label it as such
4: change the & in &attr to %, thus %attr
5: Any line that doesnt start with a %, needs to
6: Remove the %{buildroot} path from each line (I use the ! so I dont have
to worry about escaping the slashes)
7: write to file

And the second one does this:
1: find everything that is not a directory in $INSTALLDIR/etc and print it
to stdout as:
"<type>&config &attr(<mode>,<uid>,<gid>) /path/to/file\n"
2: If a line starts with d or l ignore it
3: Get rid of the line that is the $INSTALLDIR/etc path
4: change the & in &attr to %, thus %attr
5: Any line that starts with F, remove the f
6: Remove the %{buildroot} path from each line (I use the ! so I dont have
to worry about escaping the slashes)
7: append to the file


If anyone has a nicer method i'd love to see it. It took me a while to
clean it up to this point.

-greg

[Tim Mooney]

Post by G***@aotx.uscourts.gov

Post by Adrián Márques
I'm an absolute newbie trying to package an application with RPM for the
first time and would like to mark some files as config files.
The top directory of this aplication has several files and directories,
only two of which hold config files. Thus, I'm doing something like this
%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed twice' warnings. However,
I queried the generated rpm and didn't find anything wrong with it. All
config files where included and correctly marked as config files.
So my questions are: can I safely ignore these warnings or listing files
twice like I have causes a problem I'm not seeing now? is there a better
way to do what I want?

You could ignore them, but it is not recommended. The problem is that you
are effectively listing them twice the way you have it above. Your first
entry "/usr/local/myAppTopDir/" tells it to add that directory and
everything under it. Your next two entries and inside that first entry's
list.

My recommendation would be to just use a couple more globs:

%files
%dir /usr/local/myAppTopDir
# this should catch most non-hidden files and dirs, except those that
# start with c or other non-alpha characters.
/usr/local/myAppTopDir/[A-Za-bd-z0-9]*

#
# If you have any other files or directories that start with c, beyond
# configDir1 and configDir2, you'll need to list them or use a glob
# that catches them but not configDir? , something like
#
# /usr/local/myAppTopDir/c[A-Za-np-z0-9]*

#
%dir /usr/local/myAppTopDir/configDir1
%config /usr/local/myAppTopDir/configDir1/*

%dir /usr/local/myAppTopDir/configDir2
%config /usr/local/myAppTopDir/configDir2/*



I also personally avoid the use of auto-generated files lists.

Tim

[Adrián Márques]

Thanks a lot Greg and Tim for your answers.
They were very helpful and are much appreciated.

Adrián.

Post by G***@aotx.uscourts.gov

Post by Adrián Márques
I'm an absolute newbie trying to package an application with RPM for the
first time and would like to mark some files as config files.
The top directory of this aplication has several files and directories,
only two of which hold config files. Thus, I'm doing something like this
%files
/usr/local/myAppTopDir/
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed twice' warnings. However,
I queried the generated rpm and didn't find anything wrong with it. All
config files where included and correctly marked as config files.
So my questions are: can I safely ignore these warnings or listing files
twice like I have causes a problem I'm not seeing now? is there a better
way to do what I want?

You could ignore them, but it is not recommended. The problem is that you
are effectively listing them twice the way you have it above. Your first
entry "/usr/local/myAppTopDir/" tells it to add that directory and
everything under it. Your next two entries and inside that first entry's
list.

Post by Adrián Márques
I have been looking through the list archives so I know many of you
would advice me to explicitly list all files. I know this would take
care of this particular problem, but I don't want to do that unless I
really have to, since I find globbing significantly more practical in
this case.

I agree that globbing tends to be a bit easier to handle in some cases as
you are learning. You could list your first entry with a %doc prefix, and
then provide more specific blobbing for the rest of that first entries
contents.
Does that make sense?
Another method you can try, and based on your later comments I think you've
realized the debate-ability of auto-listing, is something like what will
%define INSTALLDIR /opt/informix
%install
<snip>
find $INSTALLDIR -printf "%y&attr(%m,%u,%g) %p\n" | grep -v
"$INSTALLDIR\/etc\/.*" | sed 's/^d/\%dir /' | sed 's/&/%/' | sed
's/^[^%]//' | sed 's!%{buildroot}!!' > %{_tmppath}/%{name}-files
find $INSTALLDIR/etc -printf "%y&config &attr(%m,%u,%g) %p\n" -not -type d
| grep -v "^[dl]" |grep -v "$INSTALLDIR\/etc$"| sed 's/&/%/g' | sed
's/^f//' | sed 's!%{buildroot}!!' >> %{_tmppath}/%{name}-files
%files -f %{_tmppath}/%{name}-files
1: find everything in the $INSTALLDIR and print it to stdout as
"<type>&attr(<mode>,<uid>,<gid>) /path/to/file\n"
2: Ignore all files in my $INSTALLDIR/etc directory cause I want to specify
them later as config files
3: If a line starts with d its a directory, so label it as such
4: change the & in &attr to %, thus %attr
5: Any line that doesnt start with a %, needs to
6: Remove the %{buildroot} path from each line (I use the ! so I dont have
to worry about escaping the slashes)
7: write to file
1: find everything that is not a directory in $INSTALLDIR/etc and print it
"<type>&config &attr(<mode>,<uid>,<gid>) /path/to/file\n"
2: If a line starts with d or l ignore it
3: Get rid of the line that is the $INSTALLDIR/etc path
4: change the & in &attr to %, thus %attr
5: Any line that starts with F, remove the f
6: Remove the %{buildroot} path from each line (I use the ! so I dont have
to worry about escaping the slashes)
7: append to the file
If anyone has a nicer method i'd love to see it. It took me a while to
clean it up to this point.
-greg
_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

[Michael A. Peters]

Post by Adrián Márques
Hi there,
I'm an absolute newbie trying to package an application with RPM for the
first time and would like to mark some files as config files.
The top directory of this aplication has several files and directories,
only two of which hold config files. Thus, I'm doing something like this
%files
/usr/local/myAppTopDir/

Just a question - any reason in particular why you are using /usr/local
instead of /opt ?

/usr/local is suppose to be for applications compiled from source on the
machine, /opt is for third party vendor packages (be they rpm or tarball
or whatever)

Post by Adrián Márques
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed twice' warnings. However,
I queried the generated rpm and didn't find anything wrong with it. All
config files where included and correctly marked as config files.
So my questions are: can I safely ignore these warnings or listing files
twice like I have causes a problem I'm not seeing now? is there a better
way to do what I want?

I usually do something like this -

echo "%%defattr(-,root,root,-)" > custom.list
for dir in `find $RPM_BUILD_ROOT/usr/local -type d`; do
fixed="`echo ${dir} |sed s?"$RPM_BUILD_ROOT"?""?`"
if [ `echo $fixed |grep -c "configDir"` -eq 0 ]; then
echo "%%dir $fixed" >> custom.list
fi
done
for file in `find RPM_BUILD_ROOT/usr/local -type f`; do
etc.

then

%files -f custome.list
%defattr(-,root,root.-)
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*

If the config files are something the sysadmin is to edit, it's
generally a good idea to do

%config(noreplace) instead of just %config

Post by Adrián Márques
I have been looking through the list archives so I know many of you
would advice me to explicitly list all files. I know this would take
care of this particular problem, but I don't want to do that unless I
really have to, since I find globbing significantly more practical in
this case.
I thank you already for taking the time to read this.
Regards.
Adrián.
_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

[Adrian Marques]

Thanks a lot Michael for your answers.
I was using /usr/local out of sheer ignorance of the fact that /opt was
a better choice in this case, so your comment in this regard was very
helpful.

Regards.

Adrián.

Post by Michael A. Peters

Post by Adrián Márques
Hi there,
I'm an absolute newbie trying to package an application with RPM for
the first time and would like to mark some files as config files.
The top directory of this aplication has several files and
directories, only two of which hold config files. Thus, I'm doing
%files
/usr/local/myAppTopDir/

Just a question - any reason in particular why you are using
/usr/local instead of /opt ?
/usr/local is suppose to be for applications compiled from source on
the machine, /opt is for third party vendor packages (be they rpm or
tarball or whatever)

Post by Adrián Márques
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
Obviously, this generates several 'file listed twice' warnings.
However, I queried the generated rpm and didn't find anything wrong
with it. All config files where included and correctly marked as
config files.
So my questions are: can I safely ignore these warnings or listing
files twice like I have causes a problem I'm not seeing now? is there
a better way to do what I want?

I usually do something like this -
echo "%%defattr(-,root,root,-)" > custom.list
for dir in `find $RPM_BUILD_ROOT/usr/local -type d`; do
fixed="`echo ${dir} |sed s?"$RPM_BUILD_ROOT"?""?`"
if [ `echo $fixed |grep -c "configDir"` -eq 0 ]; then
echo "%%dir $fixed" >> custom.list
fi
done
for file in `find RPM_BUILD_ROOT/usr/local -type f`; do
etc.
then
%files -f custome.list
%defattr(-,root,root.-)
%config /usr/local/myAppTopDir/configDir1/*
%config /usr/local/myAppTopDir/configDir2/*
If the config files are something the sysadmin is to edit, it's
generally a good idea to do
%config(noreplace) instead of just %config

Post by Adrián Márques
I have been looking through the list archives so I know many of you
would advice me to explicitly list all files. I know this would take
care of this particular problem, but I don't want to do that unless I
really have to, since I find globbing significantly more practical in
this case.
I thank you already for taking the time to read this.
Regards.
Adrián.
_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

[G***@aotx.uscourts.gov]

Post by Michael A. Peters

Post by Adrián Márques
%files
/usr/local/myAppTopDir/

Just a question - any reason in particular why you are using /usr/local
instead of /opt ?
/usr/local is suppose to be for applications compiled from source on the
machine, /opt is for third party vendor packages (be they rpm or tarball
or whatever)

Before I start, this is my interpretation of what I've read. I'm open to
being convinced otherwise, this is just what I've gathered over time. Okay,
and go:

This isn't really a third party vendor package if its his. In that case
its a local package that he is just packaging it for easy distribution on
his own systems. Packaging the application for easy internal deployment
does not change it from locally installed software to third party software.
I'm not saying /opt is not an acceptable location, but at the same time
/usr/local/ is just as acceptable. Per the FHS:

"The /usr/local hierarchy is for use by the system administrator when
installing software locally. It needs to be safe from being overwritten
when the system software is updated. It may be used for programs and data
that are shareable amongst a group of hosts, but not found in /usr. " [1]

"/opt is reserved for the installation of add-on application software
packages. " [2]

If one was to hold true that anything not provided by the OS belongs in
/opt, then one could argue that all of EPEL should be configured for /opt,
and back in Fedora Extras days, all of Extras should have been in /opt.

When installing into /usr/local/ you do need to make sure that your
application is not writing back to anything in that directory space.

-greg



[1] http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY
[2]
http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES

[Adrian Marques]

Thank you guys for all your comments, they are being very informative.

Just as background info, this does qualify as a third party vendor
package; the company I work for is exploring different ways in which to
distribute its application and I'm trying out RPM for this reason.

Regards.

Adrián.

Post by G***@aotx.uscourts.gov

Post by Michael A. Peters

Post by Adrián Márques
%files
/usr/local/myAppTopDir/

Just a question - any reason in particular why you are using /usr/local
instead of /opt ?
/usr/local is suppose to be for applications compiled from source on the
machine, /opt is for third party vendor packages (be they rpm or tarball
or whatever)

Before I start, this is my interpretation of what I've read. I'm open to
being convinced otherwise, this is just what I've gathered over time. Okay,
This isn't really a third party vendor package if its his. In that case
its a local package that he is just packaging it for easy distribution on
his own systems. Packaging the application for easy internal deployment
does not change it from locally installed software to third party software.
I'm not saying /opt is not an acceptable location, but at the same time
"The /usr/local hierarchy is for use by the system administrator when
installing software locally. It needs to be safe from being overwritten
when the system software is updated. It may be used for programs and data
that are shareable amongst a group of hosts, but not found in /usr. " [1]
"/opt is reserved for the installation of add-on application software
packages. " [2]
If one was to hold true that anything not provided by the OS belongs in
/opt, then one could argue that all of EPEL should be configured for /opt,
and back in Fedora Extras days, all of Extras should have been in /opt.
When installing into /usr/local/ you do need to make sure that your
application is not writing back to anything in that directory space.
-greg
[1] http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY
[2]
http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES
_______________________________________________
Rpm-list mailing list
https://www.redhat.com/mailman/listinfo/rpm-list

[G***@aotx.uscourts.gov]

Post by Adrian Marques
Just as background info, this does qualify as a third party vendor
package; the company I work for is exploring different ways in which to
distribute its application and I'm trying out RPM for this reason.

okay, so i retract it towards his, but the question still stands for local
packages if anyone would care to give input?

-greg

[Michael A. Peters]

Post by G***@aotx.uscourts.gov
If one was to hold true that anything not provided by the OS belongs in
/opt, then one could argue that all of EPEL should be configured for /opt,
and back in Fedora Extras days, all of Extras should have been in /opt.

I don't know specifically about Fedora Extras, but there have been
problems caused by third party package repositories using /usr as their
prefix.

Some or all of those issues may have been avoidable if /opt/reponame had
been used as _prefix with a single file in /etc/ld.so.conf.d/ so the
shared libraries would work (without resorting to rpath) and a single
file in /etc/profile.d/ to adjust the man path and the execution path.