============= BEFORE STARTING ============= Ok, the instructions and commands in the following sections could be used to setup a pigeonair database using the example files provided in this directory. Some of the indicated commands are *DANGEROUS*. Run them only at your own risk and only if you can really understand what they do and what I'm telling you! ============= PRECONDITIONS *read them carefully* ============= - you need a system with slapd up and running, and with all the openldap tools in your PATH. (if they are not don't worry, just keep that in mind when typing the commands). While writing the following sections, I made the following assumptions which may be wrong for your system, just adjust the corresponding commands to suit your needs: /var/lib/ldap/ -> is the directory where your ldap database is stored. /etc/init.d/slapd -> is a script you can call to start and stop your slapd installation, like: /etc/init.d/slapd start to start your slapd daemon, and /etc/init.d/slapd stop to stop your slapd daemon. /etc/ldap -> is the directory where all openldap configuration files are kept. -> You have enough privilges to manually modify and edit slapd/openldap configuration files. -> You are using bash as your shell on a GNU system. If you are not, do not worry too much, just consider that some of the following commands may need some adjustments to work on your system. ============= Configure slapd.conf ============= - remember to compile slapd with HDB support... any version > 2.2 should support it. We recommend > 2.2.25 but < 2.3, linked with Berkeley db 4.2.52 or 4.3 (make sure to read about the needed patches to those libraries). - mkdir /etc/ldap/pigeonair - copy pigeondeliver schema files in /etc/ldap/pigeonair cp ./schema/* /etc/ldap/pigeonair - open /etc/ldap/slapd.conf (slapd configuration file, wherever it is on your system) with some editor - add some includes to load the pigeonair schema files, right after the core.schema file is included: ## pigeonair required schemes include /etc/ldap/pigeonair/generic.schema include /etc/ldap/pigeonair/domain_user.schema ## modules schema include /etc/ldap/pigeonair/mailStore.schema include /etc/ldap/pigeonair/mailForward.schema include /etc/ldap/pigeonair/mailHidden.schema include /etc/ldap/pigeonair/mailAlias.schema include /etc/ldap/pigeonair/mailAntivirus.schema include /etc/ldap/pigeonair/mailNewsletter.schema include /etc/ldap/pigeonair/mailSanitizer.schema include /etc/ldap/pigeonair/mailVacation.schema include /etc/ldap/pigeonair/mailAntispam.schema ## pigeonphpbase include /etc/ldap/pigeonair/file_directory.schema ## pigeonreader include /etc/ldap/pigeonair/addressbook.schema include /etc/ldap/pigeonair/pigeonreader_options.schema ## pigeonadmin include /etc/ldap/pigeonair/adminDomain.schema include /etc/ldap/pigeonair/adminProfile.schema include /etc/ldap/pigeonair/lock.schema - if slapd has been compiled with dynamic module support, add the line: moduleload back_hdb to instruct slapd to load the back_hdb module. The configuration file may look something like: modulepath /usr/lib/ldap moduleload back_hdb (take a look to the more complete slapd.conf in this directory) - create a database section, with something like: database hdb suffix "dc=pippo,dc=it" # Where the database file are physically stored for database #1 directory "/var/lib/ldap" index objectClass eq index domain eq index account eq lastmod on - add an ACL, to allow the administrator to access the LDAP tree read/write: access to attrs=userPassword by dn="cn=admin,dc=pippo,dc=it" write by anonymous auth by self write by * none ============= Create the database... ============= - if it is running, stop slapd right now, with: /etc/init.d/slapd stop - remove any existing slapd database. This is very important since, by default, openldap uses the bdb backend while we are using the hdb. To do so, run something like: # shopt -s extglob # cd /var/lib/ldap # rm !(DB_CONFIG) which roughly means: enable bash extended glob expressions, go into the /var/lib/ldap directory, remove all files but not DB_CONFIG. If you don't have bash, please make sure to remove all files _but not_ the DB_CONFIG file. - if you don't have a DB_CONFIG file, this is probably the right time to create one. Please refer to the example DB_CONFIG file provided, and search the openldap faqs and mailing lists. *** AN UNPROPERLY CONFIGURED DB_CONFIG FILE may HANG your SLAPD and CORRUPT your DATA!! *** (this is not always true with recent versions of slapd and bdb, but it has been certainly true for a long while). - now, you should be able to create your base database: slapadd < pippo.it-base.ldiff and to create the first account: slapadd < cn=admin,dc=pippo,dc=it-base.ldiff (remember, the password is: pippo -- read the README!) - if slapd failed, it may be well because there are errors in your slapd.conf file. Please make sure it is in good shape! Eventually, test it using the slaptest command. Did you forget the schema files? did you leave any unused database configuration? did you remove the old database before creating the new one? did you compile slapd with hdb support? ... - start slapd, since we should now be almost done: /etc/init.d/slapd start - now, the database has been started with a basic ldap tree. To add a domain, you can use the file pluto.it-domain.ldiff. This file describes a simple domain pluto.it inserted in the pippo.it DSCM hierarchy, without any option or module specified. To add this file, use something like: ldapadd -x -D cn=admin,dc=pippo,dc=it -w pippo \ < pluto.it-domain.ldiff Which authenticates you as (-D) admin of pippo.it, using simple authentication (-x) and 'pippo' as password (-w, use -W not to type the password on the command line, which is usually a good idea). - to add some modules under this domain, use the same command as above, and the files named like pluto.it-modulename.ldiff - to add a user and its modules to this domain, use the files named like: pluto.it-mailUsers.ldiff (to tell DSCM the domain actually has users) user@pluto.it.ldiff (to add the user to the domain) user@pluto.it-modulename.ldiff (to add a module to the user) - enjoy yourself! ============= Notes about modifying the supplied example files... ============= Modifying the example files should be easy enough if you have some experience with LDAP directories. If you don't, just always remember: - to change both the dn of an entry and the corresponding attribute in the entry. For example, if you modify the dn of the first entry in pippo.it-base.ldiff, also remember to change the dc attribute, wich should be changed from dc: pippo to whatever you specified in the dn. - if you change the dn of an entry with children, you than need to change all the dns of the children entries. Just take your time and do it with patience. - there may be more/less attributes you can specify in the ldiff file. Just read the schema for the objectClass you are modifying in order to verify you don't need more/less attributes than specified. - to modify the passwords (userPassword attribute) manually, before inserting the ldiff file in your database and before being able to use standard tools, you can do something like: - htpasswd -n user type your password twice, and get a string like: user:yQdQq56PWHFZs - copy the part after the : (yQ...Zs), and run the command: print -MMIME::Base64 -e \ 'print encode_base64("{crypt}yQdQq56PWHFZs");' - This should ouput you something like e2NyeXB0fXlRZFFxNTZQV0hGWnM= just copy your encrypted password, and paste it in your ldiff file. - all the attributes (fields) followed by a double colon (::) are printed encoded in base64 by slap../ldap tools. To decode them you need to use something like: perl -MMIME::Base64 -pi -e \ 's/(.*)\n/decode_base64($1) . "\n"/e' Note that sometimes pigeonair needs to store mime64 encoded data in the ldap directory, wich is printed mime64 encoded by those tools. To actually take a look to the value of those fields, you need to decode the data twice. ============= Other useful commands/files/notes... ============= - in ldap terms, there is the * and + wildcard. - * will return all ``fields'' of a given record, but not ``administrative'' ones. - + will return all ``system fields'', those fields like the modifyTime or alike which are kept by slapd to handle the record. - you can use ldapsearch to search your database (man ldapsearch). Usually, something like $ ldapsearch -x -b dc=pippo,dc=it (the -x is important .. it tells ldapsearch to use simple authentication). If you want a specific attribute, just use: $ ldapsearch -x -b dc=pippo,dc=it cn in this case, ldapsearch will return just the cn attribute of every record. If you want to search another host: $ ldapsearch -x -h www.server2.org \ -b dc=pippo,dc=it cn where www.server2.org is the name of the host. You can also specify filters, with something like: $ ldapsearch -x -b dc=pippo,dc=it \ '(objectClass=mailDomain)' domain for example, to retrieve the list of domains. Note that there is a hard limit on the number of results a given ldap server will return (man slapd.conf). If you want to authenticate, in order to even access more attributes of the ldap objects, use something like: $ ldapsearch -x -w pippo -D \ cn=admin,dc=pippo,dc=it \ -b dc=pippo,dc=it \ '(objectClass=mailDomain)' domain if you don't want to put your password on the command line, use the -W parameter, instead of -w, which will ask you to type the password. - if you need to change an entry in the LDAP database, you can use: ldapmodify -x -D cn=admin,dc=pippo,dc=it -W you can use one of the -enable/disable.ldiff files as input to ldapmodify. For example, to disable the mailVacation module for pluto.it, you can: ldapmodify -x -D \ cn=admin,dc=pippo,dc=it -W \ < pluto.it-mailVacation-disable.ldiff ============= Notes! ALWAYS REMEMBER... ============= - slap* tools (like slapadd and slapcat) are privileged, and access your openldap files directly from the hard drive, often printing/handling unneeded details and without providing useful filtering capabilities. - ldap* tools (like ldapsearch and ldapadd) are not privileged. They access your LDAP directory by using the LDAP protocol, providing filtering capabilities and appliying the configured ACLs and limits. - if you change the slapd index configuration (add or remove slapd.conf index directives), always stop slapd, run the command: # slapindex and then restart slapd. - when using ldapadd, ldapmodify, ... if you get an error like: ldap_sasl_interactive_bind_s: No such attribute (16) the problem is not your ldiff file! nor the attributes therein specified! the problem is that you probably forgot the -x parameter, and you attempted sasl authentication instead of the simple one... - ldapadd, ldapmodify, ... and all openldap commands starting with ldap, use the LDAP protocol to communicate to an ldap server. They thus work: only if slapd is up and running, and only if you supply authentication data and you have enough privileges to pass the slapd ACLs/ACIs. In contrast, all db_* tools, slapadd, slapcat, slapindex and so on access directly the files on your hard drive. To use them, you need enough privileges on your system to be able to read/write slapd internal files and they can be more dangerous than the corresponding ldap* commands (when and if they exist). - when adding tons of data to a slapd server, and you are using any recent version of openldap (> 2.2.18, I believe), stop slapd and run slapadd with the -q parameter, with something like: # slapadd -q < file.ldiff the -q parameter disables transactions and checkpointing, and performs fewer input validation. This can drastically improve insert time (inserting a big database can take from a couple hours to a couple minutes). - if you stop slapd violently, always remember to either: - recreate the slapd database from a slave/backup copy - run db_recover: just go in /var/lib/ldap and type # db_recover (on some systems, since more bdb versions must live together, db_recover is called db4.2_recover or db4.3_recover -- if you use the wrong version, a warning will be outputted and the command will stop) - sometimes, it is necessary to rebuild indexes. Althought your database may be sane, sometimes db_recover/slapd puts indexes and the real database out of sync, leading to records not being found even if they are there, in the database. Just run: # slapindex while your slapd server is stopped in order to rebuild them. sometimes even after recovering a database with db_recover and after rebuilding the indexes with slapindex, slapd keeps crashing. Make sure your DB_CONFIG is set up correctly for your installation (read openldap FAQ!!) and rebuild the database from scratch, this should fix your problem... (some older slapd/bdb releases and some particular couples were _very_ broken). - to back a database up, run something like: # slapcat > backup.file whenever you like... - always remember either: - to set a checkpoint directive in your slapd.conf file _after_ the backend directive - to put db_checkpoint in your crontab - to run db_checkpoint as a daemon without it, _you will_ loose data whenever slapd is stopped uncorrectly or whenever it crashes. - get rid of the log.000* files. Unless you specify so in your DB_CONFIG file, they won't be removed automatically and left there for disaster recovery. In the best possible world, you should sometimes run: - cd /var/lib/ldap; db_checkpoint -1 - cd /var/lib/ldap; db_archive and either backup or remove the files outputted by db_archive. db_archive list files that are no longer actively used by bdb, which can be either removed or backed up for disaster recovery. (in case of disaster, you can replay them to recover your db). A good administrator should create a cron job to backup those files on some external media, and to remove them from their hard drives. NOTEs: db_checkpoint and db_archive, on your system may have a different name, like db4.2_checkpoint or db4.3_archive, depending on the bdb version being used. Also, some versions of bdb and slapd are very buggy, and are not able to close long lasting transactions (search the openldap mailing list), leading to db_archive not outputting any file until slapd is stopped, and db_checkpoint not able to save all transactions. This may be vary bad in case slapd crashes, in which case, usually, the database gets really injuried. - after running any db_* tool manually, _always_ remember to setup privileges correctly. If you are running them as root, and slapd is running under the ldap user, /var/lib/ldap/* files should be owned by the ldap user, not root. Just run something like: chown ldap.ldap /var/lib/ldap/* before restarting slapd. - enjoy your life, get as happy as you can, and share your happyness and joy with everyone you come in contact with...