19:04:48 <MerlinTHP> #startmeeting
19:04:48 <zodbot> Meeting started Sat Dec  5 19:04:48 2009 UTC.  The chair is MerlinTHP. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:04:48 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:04:57 <MerlinTHP> #topic MirrorManager
19:05:21 <MerlinTHP> Presentation by Matt Domsch on Fedora's MirrorManager system
19:07:34 <MerlinTHP> MirrorManager provides a an extensible content distribution system delivering Fedora and EPEL bits to users
19:07:52 <MerlinTHP> All done with free and open source software
19:08:05 <MerlinTHP> At least 2 million end users
19:08:12 <MerlinTHP> At least 300 mirror server admins
19:09:07 <MerlinTHP> Users want current content, fast inexpensive downloads
19:09:38 <MerlinTHP> Mirror admins want to serve local users quickly and inexpensively, and only the content they want
19:10:05 <MerlinTHP> Fedora Infrastructure Team users mirrors to reduce fedora data centre bandwidth
19:11:12 <MerlinTHP> rsync from master mirrors to local mirrors
19:11:26 <MerlinTHP> yum requests mirrors from a central director
19:12:19 <MerlinTHP> ISO downloads from download.fedoraproject.org are automatically redirected to a mirror
19:12:42 <MerlinTHP> yum uses the mirrorlist from mirrors.fedoraproject.org
19:13:24 <MerlinTHP> Mirror selection algorithm:
19:13:27 <MerlinTHP> Same network block
19:13:36 <MerlinTHP> Same Autonomous System Number
19:13:47 <MerlinTHP> Client on Internet2 gets Internet2 mirror
19:13:49 <MerlinTHP> Same country
19:13:51 <MerlinTHP> Same continent
19:13:57 <MerlinTHP> Any global mirror
19:14:09 <MerlinTHP> All lookups can be IPv4 or IPv6.
19:14:31 <MerlinTHP> Country and continent info provided by MaMind GeoLite
19:14:37 <MerlinTHP> MaxMind
19:16:20 <MerlinTHP> ASN info grabbed from internet BGP tables
19:17:56 <MerlinTHP> A fedora mirror needs:
19:18:02 <MerlinTHP> 200Gb to 1TB of disk space
19:18:18 <MerlinTHP> You need to create a Fedora Account System account
19:18:38 <MerlinTHP> Create a site in MirrorManager https://admin.fedoraproject.org/mirrormanager
19:18:52 <MerlinTHP> Fill in netblocks / AS info
19:18:57 <MerlinTHP> rsync from antoher mirror
19:19:53 <MerlinTHP> Want to recruit mirrors in china, india, russia, brazil
19:20:11 <MerlinTHP> Matt's presentation ends.
19:21:48 <MerlinTHP> Presentation on CHASM by Robert Escriva
19:22:56 <MerlinTHP> CHASM is a project designed to improve the software used for mirroring to make it possible for mirror maintainers to more easily manage advanced configs
19:23:19 <MerlinTHP> CHASM will provide:
19:23:24 <MerlinTHP> atomic updates
19:23:30 <MerlinTHP> data integrity assuranaces
19:23:42 <MerlinTHP> peer-to-peer network structure
19:23:44 <MerlinTHP> easy admin tools
19:23:47 <MerlinTHP> sane migration path
19:23:59 <MerlinTHP> CHASM is not intended to provide:
19:24:10 <MerlinTHP> some stuff that went by too quick, sorry
19:24:35 <MerlinTHP> Normal tree-type mirror distribution
19:24:49 <MerlinTHP> CHASM provides a more mesh-like structure
19:24:59 <MerlinTHP> CHASM relies on filesystem atomic operations
19:27:26 <MerlinTHP> Uses cryptographic hashing (SHA2 by default) of all content
19:28:32 <MerlinTHP> CHASM tools automatically update signatures on content update
19:31:24 <MerlinTHP> Uses a peer-to-peer network structure to cut down latency caused by tiered mirror structure
19:31:36 <MerlinTHP> Mirrors ask for content from any mirror that has it
19:31:54 <MerlinTHP> Can be configured with single-master, turning off peer-to-peer
19:32:55 <MerlinTHP> Admin tools: fsck (use the signed manifest to verify the mirror), status (check which upstream version is mirrored), update (force a manual update of the mirror)
19:33:36 <MerlinTHP> Sane migration path provided by not impacting end user and downstream rsync operation
19:35:56 <MerlinTHP> Two parts:  network daemon and staging tool
19:36:06 <MerlinTHP> ( sorry, really bad wifi lag )
19:36:49 <MerlinTHP> Plan to have a beta quality release before the end of 2009
19:37:41 * nirik wonders what it's written in/packaged? ie, can it run on various mirrors that run different os/versions?
19:40:22 <MerlinTHP> Currently prototyped in python
19:40:32 <MerlinTHP> Not averse to writing in C++ if speed neded
19:41:34 <nirik> cool. (Hopefully portable to pythons that might be on rhel4/rhel5 machines)
19:42:36 <MerlinTHP> ototyped in C++
19:42:59 <MerlinTHP> No idea where that came from, sorry
19:45:29 <MerlinTHP> Mirrors contact a central tracker to ask for peers
19:46:13 <MerlinTHP> Private mirrors can request that the tracker not tell other peers about it
19:48:36 <MerlinTHP> Intention is to make setup simple: install software, than grab a .chasm configuration seed file.
19:55:12 <MerlinTHP> Supports exclusions
19:55:23 <MerlinTHP> path-based exclusion specification
19:56:11 <MerlinTHP> #endmeeting