broken shared-mime-info and MIME typing

broken shared-mime-info and MIME typing

[Stanislav Brabec]

Looking at MIME binding, I found that it is now completely broken.

We have mime.class, that provides install-time generating of the
database.

shared-mime-info attempts to provide compile-time generating of the
database. As it does not contain freedesktop.org.xml, any package using
mime.class calling update-mime-database will remove 99% of entries in
the MIME database.

Creating database during the build time is not possible yet and would
require significant changes in update-mime-database (or writing a
different tool):
There is no way to create /usr/share/mime/magic, subclasses, mime.cache
during the build time => it must be done during installation. Note that
mime.cache is a binary file.


Proposal:

- Install freedesktop.org.xml again.
- Change mime.class:
  - Change scripts: First test, whether update-mime-database is
    installed, if yes, then call it.
  - mime class should not RDEPEND on shared-mime-info (now it depends,
    but it is missing there and may cause installation errors)
  - mime_prerm should be probably postrm => update database after
    package removal
- default to --disable-update-mimedb

Alternatives:

- Write update-mime-database alternative capable to build magic without
  freedesktop.org.xml.
- Patch update-mime-database to allow compressed packages/*.xml.
- Return to compile time and write a tool capable to create MIME
  database for all packages available in the feed as the part of
  package-index creation. It would need a special XDG_* variables
  settings, so applications will be able to collect magic from all
  repositories.

-- 
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus

Re: broken shared-mime-info and MIME typing

[Koen Kooi]

On 04-12-08 10:51, Stanislav Brabec wrote:
> Looking at MIME binding, I found that it is now completely broken.
>
> We have mime.class, that provides install-time generating of the
> database.
>
> shared-mime-info attempts to provide compile-time generating of the
> database. As it does not contain freedesktop.org.xml, any package using
> mime.class calling update-mime-database will remove 99% of entries in
> the MIME database.
>
> Creating database during the build time is not possible yet and would
> require significant changes in update-mime-database (or writing a
> different tool):
> There is no way to create /usr/share/mime/magic, subclasses, mime.cache
> during the build time =>  it must be done during installation. Note that
> mime.cache is a binary file.
>
>
> Proposal:
>
> - Install freedesktop.org.xml again.
> - Change mime.class:
>    - Change scripts: First test, whether update-mime-database is
>      installed, if yes, then call it.

Done

>    - mime_prerm should be probably postrm =>  update database after
>      package removal

Done

regards,

Koen

Re: broken shared-mime-info and MIME typing

[Koen Kooi]

On 04-12-08 10:51, Stanislav Brabec wrote:
> Looking at MIME binding, I found that it is now completely broken.
>
> We have mime.class, that provides install-time generating of the
> database.
>
> shared-mime-info attempts to provide compile-time generating of the
> database. As it does not contain freedesktop.org.xml, any package using
> mime.class calling update-mime-database will remove 99% of entries in
> the MIME database.
>
> Creating database during the build time is not possible yet and would
> require significant changes in update-mime-database (or writing a
> different tool):
> There is no way to create /usr/share/mime/magic, subclasses, mime.cache
> during the build time =>  it must be done during installation. Note that
> mime.cache is a binary file.
>
>
> Proposal:
>
> - Install freedesktop.org.xml again.
> - Change mime.class:
>    - Change scripts: First test, whether update-mime-database is
>      installed, if yes, then call it.
>    - mime class should not RDEPEND on shared-mime-info (now it depends,
>      but it is missing there and may cause installation errors)
>    - mime_prerm should be probably postrm =>  update database after
>      package removal
> - default to --disable-update-mimedb

The current implementation is:

* shared-mime-info does *NOT* include freedesktop.org.xml to save space
* mime.bbclass *DOES* check for /usr/bin/update-mime-info
* mime.bbclass *DOES* add rdepends to freedesktop.org.xml when needed

This means that people not needing freedesktop.org.xml don't get it 
installed and people that do need it get it installed automagically when 
using mime.bbclass

regards,

Koen


>
> Alternatives:
>
> - Write update-mime-database alternative capable to build magic without
>    freedesktop.org.xml.
> - Patch update-mime-database to allow compressed packages/*.xml.
> - Return to compile time and write a tool capable to create MIME
>    database for all packages available in the feed as the part of
>    package-index creation. It would need a special XDG_* variables
>    settings, so applications will be able to collect magic from all
>    repositories.
>

Re: broken shared-mime-info and MIME typing

[Stanislav Brabec]

Koen Kooi wrote:

> * shared-mime-info does *NOT* include freedesktop.org.xml to save space
> * mime.bbclass *DOES* check for /usr/bin/update-mime-info
> * mime.bbclass *DOES* add rdepends to freedesktop.org.xml when needed
> 
> This means that people not needing freedesktop.org.xml don't get it 
> installed and people that do need it get it installed automagically when 
> using mime.bbclass

NOTE: adding mime postinst and postrm scripts to libgeda
NOTE: adding shared-mime-info dependency to libgeda

This would not work. Without freedesktop.org.xml the command
"update-mime-database /usr/share/mime" will ruin the database.

I would like to propose following changes:

1. Move update-mime-database to freedesktop-mime-info.

2. RRECCOMMENDS="freedesktop-mime-info" in the class instead of RDEPENDS
   In fact, MIME database is required by its consumers, not packages
   provides XML files.

3. Implicit FILES_${PN} += "${datadir}/mime/packages"

4. Remove the warning and error in mime_.

5. shared-mime-info-locale-* should not depend on freedesktop-mime-info.
It should be probably named freedesktop-mime-info-locale-*, as
shared-mime-info itself does not need it.


-- 
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus

Re: broken shared-mime-info and MIME typing

[Koen Kooi]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]


Op 4 dec 2008, om 14:09 heeft Stanislav Brabec het volgende geschreven:

> Koen Kooi wrote:
>
>> * shared-mime-info does *NOT* include freedesktop.org.xml to save  
>> space
>> * mime.bbclass *DOES* check for /usr/bin/update-mime-info
>> * mime.bbclass *DOES* add rdepends to freedesktop.org.xml when needed
>>
>> This means that people not needing freedesktop.org.xml don't get it
>> installed and people that do need it get it installed automagically  
>> when
>> using mime.bbclass
>
> NOTE: adding mime postinst and postrm scripts to libgeda
> NOTE: adding shared-mime-info dependency to libgeda
>
> This would not work. Without freedesktop.org.xml the command
> "update-mime-database /usr/share/mime" will ruin the database.

That does work, had you looked a bit better:

mime.bbclass does:
		rdepends.append("freedesktop-mime-info")

shared-mime-info_0.51.bb does:
		RDEPENDS_freedesktop-mime-info = "shared-mime-info"

So how will that *NOT* install freedesktop.org.xml?

I tested it on 2 devices and it does the right thing:  
freedesktop.org.xml gets installed before the update-mime-database  
command is run.

FWIW: this is a subscriber only list, so quit with the reply-all

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 193 bytes --]

Re: broken shared-mime-info and MIME typing

[Stanislav Brabec]

Koen Kooi wrote:

> > This would not work. Without freedesktop.org.xml the command
> > "update-mime-database /usr/share/mime" will ruin the database.
> 
> That does work, had you looked a bit better:
> 
> mime.bbclass does:
> 		rdepends.append("freedesktop-mime-info")

My mistake, NOTE says different. But problem 1 is still valid for manual
calling of "update-mime-database /usr/share/mime" on a system, that does
not have freedesktop-mime-info.

2., 3. and 4. were just a suggestions, 5. is a minor issue.

See attached patch (untested), which should do 2., 3. and 4.
Issue 5 fix is not done yet.

> I tested it on 2 devices and it does the right thing:  
> freedesktop.org.xml gets installed before the update-mime-database  
> command is run.

Yes, it works for packages. Actually users of locales will get
freedesktop-mime-info automatically due to 5.

diff --git a/classes/mime.bbclass b/classes/mime.bbclass
index a214c11..75c1db9 100644
--- a/classes/mime.bbclass
+++ b/classes/mime.bbclass
@@ -5,9 +5,6 @@ if [ "$1" = configure ]; then
 	if [ -x ${bindir}/update-mime-database ] ; then
 		echo "Updating MIME database... this may take a while."
 		update-mime-database $D${datadir}/mime
-	else
-		echo "Missing ${bindir}/update-mime-database, update of mime database failed!"
-		exit 1
 	fi
 fi
 }
@@ -17,9 +14,6 @@ if [ "$1" = remove ] || [ "$1" = upgrade ]; then
 	if [ -x ${bindir}/update-mime-database ] ; then
 		echo "Updating MIME database... this may take a while."
 		update-mime-database $D${datadir}/mime
-	else
-		echo "Missing ${bindir}/update-mime-database, update of mime database failed!"
-		exit 1
 	fi
 fi
 }
@@ -50,7 +44,9 @@ python populate_packages_append () {
 			postrm += bb.data.getVar('mime_postrm', d, 1)
 			bb.data.setVar('pkg_postrm_%s' % pkg, postrm, d)
 			bb.note("adding shared-mime-info dependency to %s" % pkg)
-			rdepends = explode_deps(bb.data.getVar('RDEPENDS_' + pkg, d, 0) or bb.data.getVar('RDEPENDS', d, 0) or "")
-			rdepends.append("freedesktop-mime-info")
-			bb.data.setVar('RDEPENDS_' + pkg, " " + " ".join(rdepends), d)
+			rrecommends = explode_deps(bb.data.getVar('RRECOMMENDS_' + pkg, d, 0) or bb.data.getVar('RRECOMMENDS', d, 0) or "")
+			rrecommends.append("freedesktop-mime-info")
+			bb.data.setVar('RRECOMMENDS_' + pkg, " " + " ".join(rrecommends), d)
 }
+
+FILES_${PN} += "${datadir}/mime/packages"
diff --git a/packages/shared-mime-info/shared-mime-info_0.51.bb b/packages/shared-mime-info/shared-mime-info_0.51.bb
index 31a7df1..cb1f329 100644
--- a/packages/shared-mime-info/shared-mime-info_0.51.bb
+++ b/packages/shared-mime-info/shared-mime-info_0.51.bb
@@ -10,6 +10,6 @@ do_install_append() {
 # freedesktop.org.xml is huge and only needed when updating the db
 # mime.bbclass will add the dependency on it automagically
 PACKAGES =+ "freedesktop-mime-info"
-FILES_freedesktop-mime-info = "${datadir}/mime/packages/freedesktop.org.xml"
+FILES_freedesktop-mime-info = "${datadir}/mime/packages/freedesktop.org.xml ${bindir}/update-mime-database"
 RDEPENDS_freedesktop-mime-info = "shared-mime-info"
 

-- 
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus

Re: broken shared-mime-info and MIME typing

[Koen Kooi]

On 04-12-08 16:33, Stanislav Brabec wrote:
> Koen Kooi wrote:
>
>>> This would not work. Without freedesktop.org.xml the command
>>> "update-mime-database /usr/share/mime" will ruin the database.
>> That does work, had you looked a bit better:
>>
>> mime.bbclass does:
>> 		rdepends.append("freedesktop-mime-info")
>
> My mistake, NOTE says different. But problem 1 is still valid for manual
> calling of "update-mime-database /usr/share/mime" on a system, that does
> not have freedesktop-mime-info.
>
> 2., 3. and 4. were just a suggestions, 5. is a minor issue.
>
> See attached patch (untested), which should do 2., 3. and 4.
> Issue 5 fix is not done yet.
>
>> I tested it on 2 devices and it does the right thing:
>> freedesktop.org.xml gets installed before the update-mime-database
>> command is run.
>
> Yes, it works for packages. Actually users of locales will get
> freedesktop-mime-info automatically due to 5.
>
> diff --git a/classes/mime.bbclass b/classes/mime.bbclass
> index a214c11..75c1db9 100644
> --- a/classes/mime.bbclass
> +++ b/classes/mime.bbclass
> @@ -5,9 +5,6 @@ if [ "$1" = configure ]; then
>   	if [ -x ${bindir}/update-mime-database ] ; then
>   		echo "Updating MIME database... this may take a while."
>   		update-mime-database $D${datadir}/mime
> -	else
> -		echo "Missing ${bindir}/update-mime-database, update of mime database failed!"
> -		exit 1
>   	fi
>   fi
>   }
> @@ -17,9 +14,6 @@ if [ "$1" = remove ] || [ "$1" = upgrade ]; then
>   	if [ -x ${bindir}/update-mime-database ] ; then
>   		echo "Updating MIME database... this may take a while."
>   		update-mime-database $D${datadir}/mime
> -	else
> -		echo "Missing ${bindir}/update-mime-database, update of mime database failed!"
> -		exit 1
>   	fi
>   fi
>   }

You want it to error out of u-m-d is not found, since the purpose of the 
postinst (updating the database) couldn't be done. Do you have a 
use-case for this non-error behaviour?


> @@ -50,7 +44,9 @@ python populate_packages_append () {
>   			postrm += bb.data.getVar('mime_postrm', d, 1)
>   			bb.data.setVar('pkg_postrm_%s' % pkg, postrm, d)
>   			bb.note("adding shared-mime-info dependency to %s" % pkg)
> -			rdepends = explode_deps(bb.data.getVar('RDEPENDS_' + pkg, d, 0) or bb.data.getVar('RDEPENDS', d, 0) or "")
> -			rdepends.append("freedesktop-mime-info")
> -			bb.data.setVar('RDEPENDS_' + pkg, " " + " ".join(rdepends), d)
> +			rrecommends = explode_deps(bb.data.getVar('RRECOMMENDS_' + pkg, d, 0) or bb.data.getVar('RRECOMMENDS', d, 0) or "")
> +			rrecommends.append("freedesktop-mime-info")
> +			bb.data.setVar('RRECOMMENDS_' + pkg, " " + " ".join(rrecommends), d)

I'm not sure what the above accomplished, apart from making it easier to 
shoot yourself in the foot. What's the usecase for allowing to install 
the xml files, but not the update tool (assuming you applied the changes 
below)?

> +FILES_${PN} += "${datadir}/mime/packages"

I'm not so sure on that one, but I guess recipes can put the package 
theu want the xml file in before ${PN} in PACKAGES


> diff --git a/packages/shared-mime-info/shared-mime-info_0.51.bb b/packages/shared-mime-info/shared-mime-info_0.51.bb
> index 31a7df1..cb1f329 100644
> --- a/packages/shared-mime-info/shared-mime-info_0.51.bb
> +++ b/packages/shared-mime-info/shared-mime-info_0.51.bb
> @@ -10,6 +10,6 @@ do_install_append() {
>   # freedesktop.org.xml is huge and only needed when updating the db
>   # mime.bbclass will add the dependency on it automagically
>   PACKAGES =+ "freedesktop-mime-info"
> -FILES_freedesktop-mime-info = "${datadir}/mime/packages/freedesktop.org.xml"
> +FILES_freedesktop-mime-info = "${datadir}/mime/packages/freedesktop.org.xml ${bindir}/update-mime-database"
>   RDEPENDS_freedesktop-mime-info = "shared-mime-info"

If you're going to package those together you should name the package 
'update-mime-database' to avoid confusion when people want to install 
the utils and do 'opkg list | grep mime'.

regards,

Koen

Re: broken shared-mime-info and MIME typing

[Stanislav Brabec]

Koen Kooi wrote:

> You want it to error out of u-m-d is not found, since the purpose of the 
> postinst (updating the database) couldn't be done. Do you have a 
> use-case for this non-error behaviour?

> I'm not sure what the above accomplished, apart from making it easier to 
> shoot yourself in the foot. What's the usecase for allowing to install 
> the xml files, but not the update tool (assuming you applied the changes 
> below)?

These XML files are important for consumers (typically file managers of
desktop systems), not for providers (packages providing a description
for its own obscure file format).

If you want such packages installed on an image with the desktop systems
that does not support shared-mime-info, you don't need shared-mime-info
nor freedesktop-mime-info packages and you will save ~1MB.

During installation of shared-mime-info+freedesktop-mime-info the
database will be created.

> If you're going to package those together you should name the package 
> 'update-mime-database' to avoid confusion when people want to install 
> the utils and do 'opkg list | grep mime'.

Whatever name we pick, allowing to install update-mime-database
installed without freedesktop.org.xml is a bad combination.

Having generated data files without both of them seems to be safe.

-- 
Stanislav Brabec
http://www.penguin.cz/~utx/zaurus