Apache module mod_athena manual

AthEngine

Setting this flag to On allows the mod_athena request handler to process requests. The engine defaults to Off, which bypasses post configuration checks, child init tasks (including shared memory and locks creation), and request handling. If the server is initially On by configuration, it will write this status to the engine shared memory object. The request handler checks to make sure BOTH the server conf and the engine object are set to On. However, there is currently no mechanism to update the engine object to turn it off while running. It's hard to imagine why you wouldn't just change the directive and send a HUP signal instead, but I can change this if there is a demand.

AthPath

This path is relative to the server's root directory.

AthSHMFile

This path may be treated differently by different platforms. It is used by the apr shm creation and attachment functions. You should be fine with leaving it in /tmp, or setting it to a file in your apache log directory.

AthMUXPrefix

Used to provide apr lock functions with a path. Same principle as AthSHMPath, except in this case multiple files are created, with a corresponding integer index appended. Depending on your platform, you may or may not see these files on the filesystem (APR_LOCK_MECH dependent).

AthMUXCount

Depending on your locking mechanism and your kernel limits, this variable may need to be adjusted. The default -1 triggers the engine to try to create locks for all farms and physicals in the configuration file, plus the engine. You can explicitly set it to one of ("-1", "row") for the same result. For file and posix pthread locks, the default should be fine. However on systems where SYSV semaphores are used, this will most likely be too high (FreeBSD default limit is 10). You can either increase your kernel limits to allow enough semaphores, or set AthMUXCount to a lower number like 8 or 4. It must not be less than 3. When there are fewer locks than objects to be locked, the locks are shared accross multiple objects. The ammount of CPU time spent locked is small enough that this has little impact (at least in installations of less than 50 physicals).

AthAllowMirror

Set to On to allow requests to /athena/mirror feature. Be sure to have you security controls thought out before enabling this feature. When mirroring, this must be enabled on the secondary engine, not the primary. By sending serialized farm and physical objects to the mirror application, one can mirror the complete state of the primary to the secondary. See Architectural suggestions for more information.

AthAllowDiverseMulti

The mod_athena handler can process multiple requests tagged on the end of a normal request using the "&" character. An example multiple embedded request looks like this:

Embedding is a handy way to cut down on network traffic.

AthFarm

This is how a farm is declared. A farm contains ( 0 <= x <= AthPhysicalsMax ) members. This works more or less like a standard apache directory declaration in that certain directives can appear inside the farm declaration and apply to that only that farm. The Member directive can only appear inside a farm. The others can appear in server config context, where they become the defaults for all farms unless overriden in the farm declaration. The membership and some settings of the farm can also be managed while the server is running using update requests. An example farm:

There can be up to AthFarmsMax declarations of unique farms. As the mod_athena config handler parses the farms, it generates a list of unique Members (physicals). This then becomes the list of available physicals to be moved around between farms or on which to update statistics. For this reason, you may find it useful to always have one farm that is not used, but contains all your servers. You will not be wasting memory by doing this.

Member

This directive only works inside an AthFarm declaration. It declares hostname as a physical, if it has not already been declared in a previous farm, and adds it to the list of members of the current farm. The hostname can be followed optionally by a flag On or Off to set its initial administrative status as online or offline. If this is not provided, it defaults to the value of AthDefaultPhysOn.

The configuration handler performs no validation of hostname. This will be up to mod_proxy to deal with.

AthAlgorithm

In server context, this sets the algorithm to be used for all farms who do not override it with an additional AthAlgorithm directive. An algorithm string is built by joining the type to the modifiers with a "-". In all cases a word can substituted by its initial letter. Here is the list of algorithm words:

Modifiers 0cus, 1cus, 2cus, are meant to be used for custom variables, but you can store any float you want in any of the fields and then use them how you want, disregarding the convention of the field names. No algorithm logic is tied to a field's name.

Type round-robin ignores all modifiers. It causes each request to be passed to the next physical in the member list of the farm, looping to the begin after the list end.

Type simple accepts a single modifier, eg. simple-cpu. If no modifier is provided it defaults to cpu. It will then always pick the machine with the lowest value in this field (these are floats in C, eg -0.5 < 0.1), and if there are more than one at the lowest value, it will end up grabbing the first in the list of this group of best candidates. In other words, if the same three physicals are at cpu value 0.0, it will pick the same one of those three each time this happens. This behavior can be modified with the AthAlgoHitAdds directive. Pay attention to this issue, because if your hit rate is much higher than your update frequency, then you can really beat up a physical.

Next, for each member of the farm, the weighted values of every requested field are added together. It then chooses the member with the lowest value of summed weighted fields.

Use this if you have a number of fields that matter when making load balancing decisions. Handily, it automatically places the highest influence on the field with most variability. If min and max are close together, or if most members are close to any particular value, their weighted values will have little (universally equal) affect on the summed total. For instance, in our example, if mem is maxed on all members, the custom values are close to zero, but cpu is inconsistent, then cpu will affect the decision most.

There is a side affect of this algorithm you may wish to prevent. It could be that a server has maxed out in a particular field, eg. cpu, that prevents it from doing much else, which then pushes the other fields close to zero. In this case it is likely that it will have the lowest summed weighted value. However, this field being maxed makes it a very bad candidate. You can prevent the algorithm from chosing a server in this situation by setting AthAlgoMaxExcluded. With this set to On, the server will rule out members who have a weighted field equal to 1 (a maximum) in any of the specified fields, as long as another is available. If every machine matches this condition, then it falls back to the lowest value.

AthAlgoMaxExcluded

See above AthAlgorithm (dynamic algorithm detail).

AthAlgoHitAdds

When a request comes through that is handled by mod_proxy as a reverse proxy request, and is targeted for load balancing, it becomes a candidate for this rule. If the first argument is set to a valid field type in the current farm, it will follow through with the "hit adds" logic for the specified field. After the farm's algorithm picks a physical, its matching field will be incremented by float. There can be only one field per farm that is affected by this rule. Before the request is handed back to mod_proxy for completion, the value of the specified field is incremented by AthAlgoHitAdds.

This is probably most useful in very large installations, and/or when the field statistics updates are infrequent for relevant members. For example, if you know a hit to a particular farm always results in the dramatic increase in a particular field, but you are not likely to get new statistics for that field before the next request comes in, you might want to use this. It will prevent a request that comes in before new stats are available from being sent to the member we already know is doing something (its field is inflated artificially by the value of float). As soon as new statistics are sent in, its affect is erased.

NOTE: Further experience has proven this to be an indispensible tuning for most installations. Specifically, if you are serving multi-frame pages or pages with images, then any instantaneous "page" request actually represents many hits, and you want these to be load balanced, despite the impossibility of having instantaneous knowledge of the load on target servers. I've used a setting "cpu .01" to great advantage.

If you have a server context setting for this rule, but want to cancel its affects on a farm, just set the field name to anything but a valid one (eg. OFF) and/or set the float value to zero.

AthExpectUpdate

When this directive is set to On, mod_athena will expect to get new statistics for a field specified by AthExpectUpdateField within every AthExpectTTL seconds for each member (physical) in the relevant farm. If it does not, it will set the physical to be timed out, and it will not be availale to be chosen by an algorithm in any farm. Physical status is managed in shared memory context, not in farm-specific context. If the engine begins receiving updates again, and AthExpectRecoverTTL is set to a non-zero value, it will mark the physical up after that number of seconds has passed since being marked out. This allows your server to recover gracefully.

It is critical to understand the context of this rule, because it has serious side affects. If you have the same physical as a member in two farms, but the farms have different settings for AthExpect..., failure in any of the rules will cause it to be marked out and be unavailable in all the farms in which that physical is a member. But... read on:

This rule is run only when an algorithm is run for a farm in which this rule is active. No physical will get marked out if no request is made to the farm, even if the TTL has expired. Also, if every physical has expired in a farm, and no request has ever been made to the farm, then the first request that does come will fail (or be sent the all down url). In low traffic environments, you can work around this problem by scripting a periodic call to balance the farm, either by hitting a url in the farm or by using the mod_athena balance query engine. This will force the rule check, and you will be alerted to failure more quickly (if you are monitoring it). In moderate or high traffic environments, it is not an issue. More details about using this feature, as well as work-arounds for the scope of physical status, are in the Architectural suggestions section.

AthExpectUpdateField

See AthExpectUpdate for details.

AthExpectTTL

See AthExpectUpdate for details.

AthExpectRecoverTTL

See AthExpectUpdate for details.

AthNominationPerUpdate

If you find that you are serving SO MANY requests per second that calculating choices slows you down-- wait that's absurd, just rediculous.

Anyway. Setting this causes the engine to choose a server each time statistics are updated. When a request comes in to update the stats for a physical, the engine looks up each farm it is in and runs the appropriate algorithm for the farm, and stores the result in each farm.

Then, when a reverse proxy request comes in for a farm, it checks the stored value, which will contain the best choice from the latest statistics, and use that. No calculation is performed.

Basically, this switches from per-request to per-update balancing.

AthSmartfarmEnable

Enabling this feature causes the engine to inspect requests for the cookie named in AthCookieNameSmartfarm, and to then handle whatever manipulation is requested in it.

AthCookieNameSmartfarm

Sets the name of the cookie that triggers smartfarm handling.

AthAllDownURL

When a request goes to a farm in which all members have been marked down OR offline this URL will be provided.

If the target farm is itself marked offline then no member checks are done and no algorithm is run. The request is immediately returned the AthOfflineUrl.

This allows you to distinguish between being down, a condition of system failure, and being offline, an administrative status. The status of down only applies to a physical member, since that is what determines system health. The status of offline applies to both farms and physicals.

Be aware that if a farm is NOT offline but all its member physicals are offline, then the AthAllDownURL is returned. This may seem odd, but since the facility to mark the farm itself offline exists there is no need to complicate mod_athena's code. If this bothers you, you can always write a script that checks if every member of a farm is marked offline, and then mark that farm offline.

The ability to mark physicals offline is provided primarily to allow for intelligent monitoring. You probably only want to be alerted when a machine fails unintentionally, eg. down, not when it's being restarted or upgraded or under some other non-failure maintenance. The mod_athena script "ath_monitor.sh" does exactly this-- it will ignore offline physicals.

AthOfflineURL

AthDefaultPhysOn

If a Member directive does not include its own On / Off flag, this becomes the initial state set for the physical as either online or offline.

AthDefaultFarmOn

This is the initial state for all farms, either online or offline. There is no per-farm status setting on start-up provided.

AthDefaultPhysStatusFormat

"%t (%o/%u) | %c | %n | %l | %m | %0 | %1 | %2 | %3 | %4 | %5 | %6 | %7 | %8 | %9 <%h> [%f]"

This is the format used for requests for physical status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.

AthDefaultFarmStatusFormat

"%n: (%o) algo: %a %h expect update: %e (chosen: %c)\n
   down url: %d offline url: %D\n
   members: %m\n"

This is the format used for requests for farm status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.

AthDefaultEngineStatusFormat

"path: %P down url: %d farms: %f physicals: %p nominate per update (0/1): %n\n
defaults for farms:\n
   algorithm: %a %h\n
   expect update: %e\n
   ttl: %t"

This is the format used for requests for engine status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.