Re: [meta-networking][PATCH] GeoIP: change data fetching strategy

[Joe MacDonald <Joe_MacDonald@mentor.com>]

[-- Attachment #1: Type: text/plain, Size: 5640 bytes --]

[Re: [oe] [meta-networking][PATCH] GeoIP: change data fetching strategy] On 15.01.07 (Wed 19:41) Khem Raj wrote:

> 
> > On Jan 7, 2015, at 6:19 PM, Joe MacDonald <joe_macdonald@mentor.com> wrote:
> > 
> > Since the GeoIP databases are not versioned and they essentially change
> > without notice, we cannot reasonably expect the checksums to be consistent
> 
> isn’t this like AUTOREV ?.

Actually, yeah, it is.  I didn't think this applied in this case since
we don't have a source repository for this, we can only get tarballs,
and therefore SRCREV wouldn't apply.  I may dig into that a bit more
this afternoon and see if it's applicable.

> IMO such packages should be mirrored somewhere regularly and used. if
> you maintain this recipe then please mirror the sources somewhere and
> then update them regularly, but from one build to another is just
> too frequent.

That's true, it shouldn't be every build.  As for mirroring them
somewhere, I suppose I can fetch updates and place them in a github
project, but if this is something we could place (and I can easily
update) on sources.openembedded.org, that may make more sense.

-J.

> 
> > from one build to the next.  Therefore, don't download them using the
> > usual wget fetcher, but do it manually at the end of the do_fetch stage
> > and place the uncompressed data files where the remainder of the build
> > expects they would've been placed.
> > 
> > Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com>
> > ---
> > 
> > I considered trying to use the wget fetcher and having it skip over the
> > checksum operation and, honestly, I'd still prefer to do it that way, but
> > looking around and at the fetch2 code, it doesn't appear to be able to support
> > that outside of a BB_STRICT_CHECKSUM="" setting, which we definitely don't want
> > to do here.
> > 
> > -J.
> > 
> > .../recipes-support/geoip/geoip_1.6.0.bb           | 36 +++++++++-------------
> > 1 file changed, 14 insertions(+), 22 deletions(-)
> > 
> > diff --git a/meta-networking/recipes-support/geoip/geoip_1.6.0.bb b/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > index 65e8362..71fda3b 100644
> > --- a/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > +++ b/meta-networking/recipes-support/geoip/geoip_1.6.0.bb
> > @@ -8,28 +8,9 @@ using reverse DNS lookups."
> > HOMEPAGE = "http://dev.maxmind.com/geoip/"
> > SECTION = "Development/Libraries"
> > 
> > -SRC_URI = "http://www.maxmind.com/download/geoip/api/c/GeoIP-1.6.0.tar.gz;name=tarball \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz;apply=no;name=GeoIP-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz;apply=no;name=GeoIPv6-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz;apply=no;name=GeoLiteCity-dat \
> > -           http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6-beta/GeoLiteCityv6.dat.gz;apply=no;name=GeoLiteCityv6-dat \
> > -"
> > -
> > -SRC_URI[tarball.md5sum] = "89f4cdfdab43f1d67364cd7c85bbe8ca"
> > -SRC_URI[tarball.sha256sum] = "075a0c2815cd099e9ec35c9569db716a3fefcdbb6a10dbfa1ce7c6cd48d4a635"
> > -
> > -SRC_URI[GeoIP-dat.md5sum] = "4bc1e8280fe2db0adc3fe48663b8926e"
> > -SRC_URI[GeoIP-dat.sha256sum] = "7fd7e4829aaaae2677a7975eeecd170134195e5b7e6fc7d30bf3caf34db41bcd"
> > -
> > -SRC_URI[GeoIPv6-dat.md5sum] = "aac7e6e9b141de80934ecee52daf7f56"
> > -SRC_URI[GeoIPv6-dat.sha256sum] = "126fd2953eb193e60538e30b4465610530383f7a782745cacdca5ba6825f471c"
> > -
> > -SRC_URI[GeoLiteCity-dat.md5sum] = "15a42c684c53d2309e6632a6d6e02531"
> > -SRC_URI[GeoLiteCity-dat.sha256sum] = "5ec02a6d39d545c77ec12cc30c6a8856883d8f55522fc5cd4f25af80163c6b3c"
> > -
> > -SRC_URI[GeoLiteCityv6-dat.md5sum] = "49d6ec946fa0a2575b5112a68d71f933"
> > -SRC_URI[GeoLiteCityv6-dat.sha256sum] = "7a345e6cf0e59f8ab589ff15020241f0b03342dd04cc584f814c4f4700d49405"
> > -
> > +SRC_URI = "http://www.maxmind.com/download/geoip/api/c/GeoIP-1.6.0.tar.gz"
> > +SRC_URI[md5sum] = "89f4cdfdab43f1d67364cd7c85bbe8ca"
> > +SRC_URI[sha256sum] = "075a0c2815cd099e9ec35c9569db716a3fefcdbb6a10dbfa1ce7c6cd48d4a635"
> > 
> > LICENSE = "LGPL-2.1"
> > 
> > @@ -43,6 +24,17 @@ inherit autotools
> > EXTRA_OECONF = "--disable-static               \
> >                 --disable-dependency-tracking  "
> > 
> > +do_fetch_append() {
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoIP.dat.gz > ${WORKDIR}/GeoIP.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoIPv6.dat.gz > ${WORKDIR}/GeoIPv6.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoLiteCity.dat.gz > ${WORKDIR}/GeoLiteCity.dat");
> > +    os.system("${FETCHCMD_wget} http://geolite.maxmind.com/download/geoip/database/GeoLiteCityv6.dat.gz");
> > +    os.system("gunzip -c ${DL_DIR}/GeoLiteCityv6.dat.gz > ${WORKDIR}/GeoLiteCityv6.dat");
> > +}
> > +
> > do_install() {
> >     make DESTDIR=${D} install
> >     install -d ${D}/${datadir}/GeoIP
> > -- 
> > 1.9.1
> > 
> > -- 
> > _______________________________________________
> > Openembedded-devel mailing list
> > Openembedded-devel@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-devel
> 
-- 
-Joe MacDonald.
:wq

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 501 bytes --]