========================================================= WADF - Web Application Deployment Framework (c)2006-2008 Tim Jackson (tim@timj.co.uk) ========================================================= Introduction ------------ WADF is a simple templating and deployment system combined with some conventions to make deployment of web applications more reliable. There are several key concepts and features: - Abstraction of all system (deployment)-specific information including database details, file paths etc. - includes generation of database, hostname and other details if required - Simple database management including setup of database server - PEAR Installer integration for dependency management - Webserver configuration management - includes ability to manage a local, user-controlled "micro" webserver for development on a local workstation - PHP-aware; supports both mod_php and CGI PHP installations Principles ---------- - WADF is based on having a checked-out copy of the core ("end user") application from a version control system - All other dependencies (libraries etc.) are then pulled in via the PEAR installer - "Profiles" allow varying runtime configuration between systems (e.g. developer workstation, staging server, live server) WADF is mostly dependent on PEAR and assumes you have a PEAR package.xml metadata file in the root of the application which describes, in particular, the dependencies of the application. See http://pear.php.net/manual/en/guide.developers.package2.php for more details. Requirements ------------ WADF requires that you have a recent version of PEAR. It is tested on Fedora Linux, but should probably work OK on most Linuxes and probably other UNIX variants (though the sample httpd.conf for the micro HTTP server may require adjustment). It will not work on Windows at present. ========================================================= INSTALLING WADF ========================================================= WADF is available from the PEAR channel "pear.timj.co.uk" and it is recommended you install it using PEAR. First you will need to discover the channel, like so: channel-discover pear.timj.co.uk and then install WADF as such: pear install timj/Tools_WADF ========================================================= BASIC WADF CONFIGURATION ========================================================= Read the heavily-commented wadf.conf. The intention is that a system-wide generic config file is used, which may then be overriden by, respectively: - User-specific config file (~/.wadf/config) - Specific config file for site (see the section "Additional Configuration - local_config") - Instance-specific configuration variables (see the section "The WADF Instance File") - Command line options Note that the configurations (both global, user-specific and site specific) are split into sections: [globals] [local] [dev] [staging] [live] Only "[globals]" is special. The other section names are merely identifiers and can be anything you wish as long as they match what you intend to use as "profile" names on the various servers/machines you are going to deploy to. It is strongly suggested that you stick with the values suggested above, where: - "local" is for deployment to a local workstation - "dev" is for deployment to a shared development server - "staging" is for deployment to a client-facing staging server - "live" is for deployment to a production server ========================================================= DEPLOYING A SITE WITH WADF ========================================================= Basic Setup Requirements ------------------------ - The webserver config dir (vhost_config_path) exists and is writeable - User can execute webserver_restart_cmd (may imply sudo setup etc., depending on the configuration value) - Apache is installed - Database server is installed and running - All required PHP extensions (e.g. php-mysql) are installed - User can write to /etc/hosts, if deploy_dns is set to "hosts" - SELinux permits webserver to read deployed files Webserver configuration ----------------------- In the root of site you should have (for Apache HTTPD): vhost-httpd.conf.template - virtual host template N.B. that this file may be pre-generated by PEAR if you so desire - in that case call it something like vhost-httpd.conf.src (though this can be anything) and get PEAR to install it as vhost-httpd.conf.template In this and any other file called *.template, a number of macros are available: @appref@ - app reference (see above) @deploy_path@ - the filesystem path to the client site e.g. /foo/bar/clientname/sitename. *NOT* /foo/bar/clientname/sitename/webroot. Does not have a trailing slash. (The server may/should create a random path name e.g. /sites/ab348dc to test whether the site really does have no references to a specific path) @vhostX_name@ - the HTTP host name @vhostX_interface@ - the vhost details which may be an IP, IP/port or multiples of these @pear_path@ - path to PEAR installation @codebase_path@ - path to "other" codebases/frameworks where the 'X' in vhostX_name and vhostX_interface is an integer. This allows apps that have several different vhost interfaces to be supported. You can also use custom macros which are between 3 and 50 characters, named using 0-9a-z_ or a subset thereof Server will replace macros in *.template and rename the file, removing the .template extension. Files within the site (including the vhost.conf and any source files e.g. PHP) MUST NOT reference ANY absolute paths except via macros. e.g. # vhost.conf.template ServerName @vhost1_name@ DocumentRoot @deploy_path@/webroot # php_dir will contain the dependencies pulled in via PEAR php_value include_path .:@deploy_path@/include:@php_dir@ RewriteEngine on RewriteRule /foo @deploy_path@/include/handle_url.php *Don't* include any hosting environment-specific configuration in here, e.g. logs. PHP configuration ----------------- PHP configuration varies according to the environment. If you are using mod_php (that is, PHP installed as an Apache HTTPD module), the PHP directives are typically included in the Apache config, as part of the section, for example: php_admin_flag engine on However, for PHP running as a CGI, the directives are usually stored in a separate php.ini file. WADF is aware of both types of PHP configuration and explicitly supports them via the "php_type" configuration parameter. In either case, it expects to find a file called "php.ini" in the root of the site (this will normally be pre-templated as php.ini.template). Then, when deploying a site, WADF inserts PHP directives in the appropriate place. - If "php_type" is set to "mod_php", then it converts the directives to Apache config style and inserts them in the VirtualHost. It also converts comments from the php.ini "; comment" format to the Apache "# comment" format. - If "php_type" is set to "cgi:/path/to/file" then it simply copies the ini file to "/path/to/file" ready to be used by the CGI interpreter. No matter whether you are running PHP as mod_php or CGI, you can also set additional PHP options on a per-profile basis. By setting the "php_config_location_extra" parameter to point to a php.ini file, the options from this ini file will be appended to the rest of the PHP options from the normal application PHP config. This can be useful, for example, to force "display_errors" to always be turned on for a development machine. Database configuration ---------------------- For each database that the application uses, you should store the schema in the root directory, named "schemaX.sql" where X is the database number. (For the first schema, omit X - i.e. call the file simply "schema.sql"). In your code, you should use a template file (see above) and use the following macros: @dbX_type@ - database type according to the PEAR conventions @dbX_host@ - database host @dbX_name@ - database name @dbX_user@ - database username @dbX_pass@ - database password where X is the database number (to support apps that call on multiple databases) e.g. @db1_type@://@db1_user@@@db1_host@/@db1_name@/@db1_pass@ This will be replaced with the database hostname, which is selectable by the WADF client according to the profile in use. The above definitions are configured separately on the staging server, on a per-application basis. ========================================================= ADDITIONAL CONFIGURATION ========================================================= Although the default configuration options are probably sufficient for deployment of sites for development purposes (e.g. using auto-generated usernames and passwords), more complex configuration is required to manage the demands of a multi-stage environment where you may have (for example) development, staging and live environments that differ. local_config ------------ The local_config option specifies the location of a file where configuration directives pertaining to the specific application (rather than applications in general) can be put. By default this is in @deploy_path@/wadf.conf. The format of this file is exactly the same as other WADF config files and typically this will include several profile sections, containing environment-specific configs (such as database hostnames, virtual host hostnames etc.). It may also contain configuration variables which are specific to that application and which are not standard WADF variables. For example: [globals] ; This is an option which is specific to this application and not a WADF ; standard variable my_config_option = foo [live] vhost1_name = www.example.com db1_host = @underscore:appref@.db.example.com my_config_option = bar [staging] vhost1_name = staging.example.com Sensitive configuration parameters ---------------------------------- There may be some configuration parameters which you regard as sensitive and may not want to store in the version control system along with the main application, for reasons of security or privacy. For example, database passwords for your live servers. In that case, you may want the person deploying the application to have to enter them. To achieve this, you can specify "%%" as the value of any configuration parameter. Then, when wadf-deploy is run, it will prompt for the value of this option. For example: [live] db1_pass = %% In the above example, whe - sample config is set up in .wadf/httpd.confn deploying to the live server, WADF will prompt as follows: "Please enter the value for the configuration option 'db1_pass':" The value stated will be stored locally in the instance file (see next section). Then, next time a wadf-reprocess is run *on that copy*, the value that was entered will be automatically substituted. If you want to edit the value, see the instructions in the next section. ========================================================= THE WADF INSTANCE FILE ========================================================= In order to know that a particular directory is a WADF-deployed application, WADF creates a special file called .wadf-instance in the deployed directory. This file is a simple newline-separated text file which has the following format: - The first line contains the application reference of the deployed application - Second and subsequent lines contain WADF configuration specific to that particular instance (typically, those entered via the "%%" configuration value construct), in the format "key = value". No comments are permitted in the file. There are no profile sections in the file because, by definition, the file is specific to the current deployed instance, whatever that may be. ========================================================= THE WADF MICRO HTTP SERVER ========================================================= WADF can start up local instances of Apache running as the current user - meaning that a complete development environment that closely simulates a live environment can be set up without requiring special privileges. You do not have to use the WADF micro HTTP server at all, if you don't want to. If you do, it is controlled via the "wadf-httpd" script which is described fully in the next section, "Command Line Usage". You will need to set the following WADF options: webserver_restart_cmd = wadf-httpd reloadstart vhost_config_path = /home/@user@/.wadf/vhosts (NB this path is currently "special" - do not change it) When first running the wadf-httpd server, a sample config is set up in ~/.wadf/httpd.conf. You can then amend this file as you see fit. After starting the WADF HTTP server, a number of additional files will be generated in ~/.wadf by default (that is, assuming you don't change the default WADF Apache config): - httpd.error_log - this is the Apache output error log - When restarting WADF, this is cleared and the old log moved to httpd.error_log.old - httpd.lock - this is just a lock file for the server which you can ignore - httpd.pid - a file containing the running process ID of the server. Leave this alone as it is used by the wadf-httpd script. - default_error_page.html - a file with a list of the deployed applications, shown if you access the local WADF HTTP server over an unconfigured interface - vhosts/00-default.html - the virtual host to trigger the default error page ========================================================= COMMAND LINE USAGE ========================================================= WADF has a number of command line tools: wadf-deploy [-d ] [] ------------------------------------------------ This performs a fresh deployment of application from the version control system. Optionally, it may be passed the following parameters: : The version control revision to retrieve. This should be in the format: tag/XXX: Use the version of the software identified with tag "XXX" branch/XXX: Use the HEAD of the branch called "XXX" trunk (default): Use the software trunk -d : A macro override -r : The specific revision number to check out. Only applicable where is set to "trunk" or "branch". wadf-reprocess -------------- Reprocess the current working directory as a WADF site (e.g. remake templates). This does *not* re-run the kickstart script. Optionally, it can be passed parameters to switch the working copy to a different version from the version control system. These take the following form: tag/XXX: Use the version of the software identified with tag "XXX" branch/XXX: Use the HEAD of the branch called "XXX" trunk: Use the software trunk wadf-clean ---------- Remove files generated from templates from the working directory wadf-httpd ------------------- For workstation-based development, controls the Apache HTTPD webserver running as the current user. is one of the following: - start: Start the webserver - stop: Stop the webserver - restart: Stop, then start the webserver - condrestart: Restart the webserver if it is already running - reload or graceful: Gracefully restart the webserver if it is already running - reloadstart: Gracefully restart the webserver if it is already running, otherwise start - status: Show status information about whether the webserver is running and if so, what applications are deployed - configtest: Check the syntax of the Apache configuration, incorporating the various virtual hosts The output of "wadf-httpd status" looks something like the following: ******************************************* Configured for the following applications: someapp: (/path/to/someapp) ver=DEVTR:1234 http://someapp.mypc.example.com:10080 otherapp: (/path/to/otherap) ver=DEVTR:4870 http://otherapp.mypc.example.com:10080 ******************************************* The "ver=" output shows which version was deployed the last time that "wadf-reprocess" was run. This is in the format described in the section "Version identifiers". ========================================================= VERSION IDENTIFIERS ========================================================= A version identifier is available to uniquely identify the version of the end-application (client application) checked out from the version control system. This is available in the 'deploy_version' macro during template processing. It is in the following format: Trunk: DEVTR:[revision] Branch: DEVBR/[branch name]:[revision] Tag: [tag name] where "[revision]" is the revision number ========================================================= CRONTAB SUPPORT ========================================================= By creating a file called "crontab" in the root of the deployed application (in standard crontab format), WADF can install standard cronjobs for the current user account. It places special metadata markers in the installed crontab so that if further redeployments are done, the crontab entries are not duplicated. ========================================================= GOTCHAS ========================================================= - If you are letting WADF generate a "clean" PEAR installation when you deploy an application, then if you need to run PEAR manually for any reason, you should be sure to use the PEAR "binary" from the "clean" install i.e. DON'T do this: pear -c /path/to/myapp/pear_local/.pearrc [something] Instead, do this: /path/to/myapp/pear_local/pear [something] Otherwise, "odd" stuff may happen with config variables e.g. applicationcore_dir ========================================================= FAQ ========================================================= Q: What about SSL? A: SSL is a hosting-environment-specific (not application-specific) configuration so should either be done via an SSL terminator or via vhost_config_append in the 'live' profile Q: What about redirects (e.g. to force a certain page to redirect to a secure version) based on whether the site is being accessed over SSL? A: Use a local_config file (see section "Additional Configuration") to set a custom variable (e.g. "enable_ssl") on a per-profile basis. Set it to "0" in the "globals" profile and then for the profile(s) that you want it enabled (e.g. "live"), set it to "1". Then, you may have a page something like this: checkout.php.template: