Terms

Directives


top

AthEngine

Description:Enables / disables request handling by amodule
Syntax:AthEngine { On | Off }
Default:AthEngine Off
Context:server config
Status:Extension
Module:mod_athena

Setting this flag to On allows the mod_athena request handler to process requests. The engine defaults to Off, which bypasses post configuration checks, child init tasks (including shared memory and locks creation), and request handling. If the server is initially On by configuration, it will write this status to the engine shared memory object. The request handler checks to make sure BOTH the server conf and the engine object are set to On. However, there is currently no mechanism to update the engine object to turn it off while running. It's hard to imagine why you wouldn't just change the directive and send a HUP signal instead, but I can change this if there is a demand.

top

AthPath

Description:
Syntax:AthPath local_url_path
Default:AthPath /athena
Context:server config
Status:Extension
Module:mod_athena

This path is relative to the server's root directory.

AthPath /my_athena


sets the url for server "my_server" mod_athena engine to:


http://myserver/my_athena

This is the url used to interact with the mod_athena engine. Requests that match this initial string will be intercepted and their handling will end with mod_athena's handler (they will not fall through to mod_proxy). The engine's functions are accessed via a directory syntax:

/athena/balance

/athena/status/farm

/athena/status/phys

/athena/status/engine

/athena/update/farm

/athena/update/phys

/athena/mirror/farm

/athena/mirror/phys

These must always be followed by an appropriate query string. Details for these features are found in Sending statistics, Runtime management, and Querying the balancer

top

AthSHMFile

Description:Path used to create and attach shared memory
Syntax:AthSHMFile file-path
Default:AthSHMFile /tmp/.apache_mod_athena_shm
Context:server config
Status:Extension
Module:mod_athena

This path may be treated differently by different platforms. It is used by the apr shm creation and attachment functions. You should be fine with leaving it in /tmp, or setting it to a file in your apache log directory.

top

AthMUXPrefix

Description:Path prefixed to locks for creation and initialization
Syntax:AthMUXPrefix file-path
Default:AthMUXPrefix /tmp/.apache_mod_athena_mux.
Context:server config
Status:Extension
Module:mod_athena

Used to provide apr lock functions with a path. Same principle as AthSHMPath, except in this case multiple files are created, with a corresponding integer index appended. Depending on your platform, you may or may not see these files on the filesystem (APR_LOCK_MECH dependent).

top

AthMUXCount

Description:Number of locks created to synchronize updates
Syntax:AthMUXCount integer
Syntax:AthMUXCount -1
Context:server config
Status:Extension
Module:mod_athena

Depending on your locking mechanism and your kernel limits, this variable may need to be adjusted. The default -1 triggers the engine to try to create locks for all farms and physicals in the configuration file, plus the engine. You can explicitly set it to one of ("-1", "row") for the same result. For file and posix pthread locks, the default should be fine. However on systems where SYSV semaphores are used, this will most likely be too high (FreeBSD default limit is 10). You can either increase your kernel limits to allow enough semaphores, or set AthMUXCount to a lower number like 8 or 4. It must not be less than 3. When there are fewer locks than objects to be locked, the locks are shared accross multiple objects. The ammount of CPU time spent locked is small enough that this has little impact (at least in installations of less than 50 physicals).

top

AthAllowMirror

Description:Controls access to mirroring features
Syntax:AthAllowMirror { On | Off }
Default:AthAllowMirror Off
Context:server config
Status:Extension
Module:mod_athena

Set to On to allow requests to /athena/mirror feature. Be sure to have you security controls thought out before enabling this feature. When mirroring, this must be enabled on the secondary engine, not the primary. By sending serialized farm and physical objects to the mirror application, one can mirror the complete state of the primary to the secondary. See Architectural suggestions for more information.

top

AthAllowDiverseMulti

Description:Enables embedding of different request types in a single request
Syntax:AthAllowDiverseMulti { On | Off }
Default:AthAllowDiverseMulti Off
Context:server config
Status:Extension
Module:mod_athena

The mod_athena handler can process multiple requests tagged on the end of a normal request using the "&" character. An example multiple embedded request looks like this:

http://my_server/athena/status/phys/?h=app1&/status/farm/?n=farm1


returns something like:


1060131735041539 (1/1) | 1.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 | 0.000000 | 0.014394 <app1> [farm1 farm2]
farm1: (1) algo: simple-cpu expect update: 1 (chosen: N/A)
   down url: http://myserver/down.html
   members: app1 app2 app3

in other words, it's handling two requests sequentially, and concatenating their output in a single reponse. Normally, for security reasons, you are only allowed to embed multiple requests if they are all for the same engine component as defined by subdirectory, eg. all "/status", "/update", "/mirror", etc. can only be embedded with their own type. AthAllowDiverseMulti allows you to embed different types in the same request when set to on, eg. a status and an update together.

Embedding is a handy way to cut down on network traffic.

top

AthFarm

Description:A collection of physicals into one entity for balancing
Syntax:AthFarm name*
*Restricted to lower case, single word. Multiple declarations allowed.
Context:server config
Status:Extension
Module:mod_athena

This is how a farm is declared. A farm contains ( 0 <= x <= AthPhysicalsMax ) members. This works more or less like a standard apache directory declaration in that certain directives can appear inside the farm declaration and apply to that only that farm. The Member directive can only appear inside a farm. The others can appear in server config context, where they become the defaults for all farms unless overriden in the farm declaration. The membership and some settings of the farm can also be managed while the server is running using update requests. An example farm:

<AthFarm farm-one>
  AthExpectUpdate On
  AthAlgorithm simple-cpu
  Member server-1
  Member server-2
  Member server-3
</AthFarm>
Because of the way mod_proxy and/or httpd processes the host component of the url in the request (it gets canonicallized) before mod_athena's handler gets it, but after mod_proxy rewrites it, farm names must be like hostnames in their canonicallized form. The simplest way to accomplish this is to just use lower case, single word farm names. I may make my farm name matching also perform canonicalization in the future, but this is the cleanest and simplest solution.

There can be up to AthFarmsMax declarations of unique farms. As the mod_athena config handler parses the farms, it generates a list of unique Members (physicals). This then becomes the list of available physicals to be moved around between farms or on which to update statistics. For this reason, you may find it useful to always have one farm that is not used, but contains all your servers. You will not be wasting memory by doing this.

top

Member

Description:A server available to a farm
Syntax:Member hostname [ On | Off ]
Context:AthFarm config
Status:Extension
Module:mod_athena

This directive only works inside an AthFarm declaration. It declares hostname as a physical, if it has not already been declared in a previous farm, and adds it to the list of members of the current farm. The hostname can be followed optionally by a flag On or Off to set its initial administrative status as online or offline. If this is not provided, it defaults to the value of AthDefaultPhysOn.

The configuration handler performs no validation of hostname. This will be up to mod_proxy to deal with.

top

AthAlgorithm

Description:A string that describes an algorithm to be used
Syntax:AthAlgorithm { r | s-c ... | d-c-n-2 ... }
Default:AthAlgorithm round-robin
Context:server, AthFarm config
Overrides:AthFarm over server
Status:Extension
Module:mod_athena

In server context, this sets the algorithm to be used for all farms who do not override it with an additional AthAlgorithm directive. An algorithm string is built by joining the type to the modifiers with a "-". In all cases a word can substituted by its initial letter. Here is the list of algorithm words:

types:
  round-robin
  simple
  dynamic
modifiers:
  cpu, net, mem, ld, disk
  0cus, 1cus, 2cus, ..., 9cus

Modifiers 0cus, 1cus, 2cus, are meant to be used for custom variables, but you can store any float you want in any of the fields and then use them how you want, disregarding the convention of the field names. No algorithm logic is tied to a field's name.

Type round-robin ignores all modifiers. It causes each request to be passed to the next physical in the member list of the farm, looping to the begin after the list end.

Type simple accepts a single modifier, eg. simple-cpu. If no modifier is provided it defaults to cpu. It will then always pick the machine with the lowest value in this field (these are floats in C, eg -0.5 < 0.1), and if there are more than one at the lowest value, it will end up grabbing the first in the list of this group of best candidates. In other words, if the same three physicals are at cpu value 0.0, it will pick the same one of those three each time this happens. This behavior can be modified with the AthAlgoHitAdds directive. Pay attention to this issue, because if your hit rate is much higher than your update frequency, then you can really beat up a physical.

Type dynamic accepts any subset of the modifiers, eg. dynamic-cpu-mem-1cus-2cus. This algorithm is a bit complicated, but still very fast. First, it traverses all members of the farm, finding the minimum and maximum values of the first field to be used, in our example, "cpu". Then, it uses this to calculate a new value for this field for each member farm, weighting it between 0 (the min) and 1 (the max). It repeats this for every requested field.

Next, for each member of the farm, the weighted values of every requested field are added together. It then chooses the member with the lowest value of summed weighted fields.

Use this if you have a number of fields that matter when making load balancing decisions. Handily, it automatically places the highest influence on the field with most variability. If min and max are close together, or if most members are close to any particular value, their weighted values will have little (universally equal) affect on the summed total. For instance, in our example, if mem is maxed on all members, the custom values are close to zero, but cpu is inconsistent, then cpu will affect the decision most.

There is a side affect of this algorithm you may wish to prevent. It could be that a server has maxed out in a particular field, eg. cpu, that prevents it from doing much else, which then pushes the other fields close to zero. In this case it is likely that it will have the lowest summed weighted value. However, this field being maxed makes it a very bad candidate. You can prevent the algorithm from chosing a server in this situation by setting AthAlgoMaxExcluded. With this set to On, the server will rule out members who have a weighted field equal to 1 (a maximum) in any of the specified fields, as long as another is available. If every machine matches this condition, then it falls back to the lowest value.

top

AthAlgoMaxExcluded

Description:Tunes the "dynamic" algorithm
Syntax:{On | Off}
Context:server, AthFarm config
Status:Extension
Module:mod_athena

See above AthAlgorithm (dynamic algorithm detail).

top

AthAlgoHitAdds

Description:A value added to a field concurrent with a reverse proxy request
Syntax:AthAlgoHitAdds { OFF | cpu | net | mem | ld | 0cus | 1cus | 2cus ... 9cus } float
Context:server, AthFarm config
Status:Extension
Module:mod_athena

When a request comes through that is handled by mod_proxy as a reverse proxy request, and is targeted for load balancing, it becomes a candidate for this rule. If the first argument is set to a valid field type in the current farm, it will follow through with the "hit adds" logic for the specified field. After the farm's algorithm picks a physical, its matching field will be incremented by float. There can be only one field per farm that is affected by this rule. Before the request is handed back to mod_proxy for completion, the value of the specified field is incremented by AthAlgoHitAdds.

This is probably most useful in very large installations, and/or when the field statistics updates are infrequent for relevant members. For example, if you know a hit to a particular farm always results in the dramatic increase in a particular field, but you are not likely to get new statistics for that field before the next request comes in, you might want to use this. It will prevent a request that comes in before new stats are available from being sent to the member we already know is doing something (its field is inflated artificially by the value of float). As soon as new statistics are sent in, its affect is erased.

NOTE: Further experience has proven this to be an indispensible tuning for most installations. Specifically, if you are serving multi-frame pages or pages with images, then any instantaneous "page" request actually represents many hits, and you want these to be load balanced, despite the impossibility of having instantaneous knowledge of the load on target servers. I've used a setting "cpu .01" to great advantage.

If you have a server context setting for this rule, but want to cancel its affects on a farm, just set the field name to anything but a valid one (eg. OFF) and/or set the float value to zero.

top

AthExpectUpdate

Description:Expect a physical statistic update within a certain amount of time
Syntax:AthExpectUpdate { On | Off }
Default:AthExpectUpdate Off
Context:server, AthFarm config
Status:Extension
Module:mod_athena

When this directive is set to On, mod_athena will expect to get new statistics for a field specified by AthExpectUpdateField within every AthExpectTTL seconds for each member (physical) in the relevant farm. If it does not, it will set the physical to be timed out, and it will not be availale to be chosen by an algorithm in any farm. Physical status is managed in shared memory context, not in farm-specific context. If the engine begins receiving updates again, and AthExpectRecoverTTL is set to a non-zero value, it will mark the physical up after that number of seconds has passed since being marked out. This allows your server to recover gracefully.

It is critical to understand the context of this rule, because it has serious side affects. If you have the same physical as a member in two farms, but the farms have different settings for AthExpect..., failure in any of the rules will cause it to be marked out and be unavailable in all the farms in which that physical is a member. But... read on:

This rule is run only when an algorithm is run for a farm in which this rule is active. No physical will get marked out if no request is made to the farm, even if the TTL has expired. Also, if every physical has expired in a farm, and no request has ever been made to the farm, then the first request that does come will fail (or be sent the all down url). In low traffic environments, you can work around this problem by scripting a periodic call to balance the farm, either by hitting a url in the farm or by using the mod_athena balance query engine. This will force the rule check, and you will be alerted to failure more quickly (if you are monitoring it). In moderate or high traffic environments, it is not an issue. More details about using this feature, as well as work-arounds for the scope of physical status, are in the Architectural suggestions section.

top

AthExpectUpdateField

Description:The field checked for updates
Syntax:AthExpectUpdateField { cpu | net | mem | ld | 0cus | 1cus ... 9cus }
Context:server, AthFarm config
Status:Extension
Module:mod_athena

See AthExpectUpdate for details.

top

AthExpectTTL

Description:The number seconds to wait for new statistics from a member
Syntax:AthExpectTTL integer, seconds
Context:server, AthFarm config
Status:Extension
Module:mod_athena

See AthExpectUpdate for details.

top

AthExpectRecoverTTL

Description:The number of seconds to wait before re-enabling a member
Syntax:AthExpectRecoverTTL integer, seconds
Context:server, AthFarm config
Status:Extension
Module:mod_athena

See AthExpectUpdate for details.

top

AthNominationPerUpdate

Description:
Syntax:AthNominationPerUpdate { On | Off }
Default:AthNominationPerUpdate Off
Context:server
Status:Extension
Module:mod_athena

If you find that you are serving SO MANY requests per second that calculating choices slows you down-- wait that's absurd, just rediculous.

Anyway. Setting this causes the engine to choose a server each time statistics are updated. When a request comes in to update the stats for a physical, the engine looks up each farm it is in and runs the appropriate algorithm for the farm, and stores the result in each farm.

Then, when a reverse proxy request comes in for a farm, it checks the stored value, which will contain the best choice from the latest statistics, and use that. No calculation is performed.

Basically, this switches from per-request to per-update balancing.

top

AthSmartfarmEnable

Description:
Syntax:AthSmartfarmEnable { On | Off }
Default:AthSmartfarmEnable Off
Context:server
Status:Extension
Module:mod_athena

Enabling this feature causes the engine to inspect requests for the cookie named in AthCookieNameSmartfarm, and to then handle whatever manipulation is requested in it.

top

AthCookieNameSmartfarm

Description:
Syntax:AthCookieNameSmartfarm cookie name
Default:AthCookieNameSmartfarm X-Ath-Smartfarm
Context:server
Status:Extension
Module:mod_athena

Sets the name of the cookie that triggers smartfarm handling.

top

AthAllDownURL

Description:Url to forward to when all farm members are down
Syntax:AthAllDownURL full/url
Context:server, AthFarm config
Status:Extension
Module:mod_athena

When a request goes to a farm in which all members have been marked down OR offline this URL will be provided.

If the target farm is itself marked offline then no member checks are done and no algorithm is run. The request is immediately returned the AthOfflineUrl.

This allows you to distinguish between being down, a condition of system failure, and being offline, an administrative status. The status of down only applies to a physical member, since that is what determines system health. The status of offline applies to both farms and physicals.

Be aware that if a farm is NOT offline but all its member physicals are offline, then the AthAllDownURL is returned. This may seem odd, but since the facility to mark the farm itself offline exists there is no need to complicate mod_athena's code. If this bothers you, you can always write a script that checks if every member of a farm is marked offline, and then mark that farm offline.

The ability to mark physicals offline is provided primarily to allow for intelligent monitoring. You probably only want to be alerted when a machine fails unintentionally, eg. down, not when it's being restarted or upgraded or under some other non-failure maintenance. The mod_athena script "ath_monitor.sh" does exactly this-- it will ignore offline physicals.

top

AthOfflineURL

Description:Url to forward to when all farm members are offline
Syntax:AthOfflineURL full/url
Context:server, AthFarm config
Status:Extension
Module:mod_athena
See AthAllDownUrl for details.

top

AthDefaultPhysOn

Description:Sets the initial adminstrative state of physicals (members)
Syntax:AthDefaultPhysOn { On | Off }
Default:AthDefaultPhysOn On
Context:server, AthFarm config
Status:Extension
Module:mod_athena

If a Member directive does not include its own On / Off flag, this becomes the initial state set for the physical as either online or offline.

top

AthDefaultFarmOn

Description:Sets the initial adminstrative state of farms
Syntax:AthDefaultFarmOn { On | Off }
Default:AthDefaultFarmOn On
Context:sever config
Status:Extension
Module:mod_athena

This is the initial state for all farms, either online or offline. There is no per-farm status setting on start-up provided.

top

AthDefaultPhysStatusFormat

Description:Sets the format string for status/phys requests
Syntax:AthDefaultPhysStatusFormat format string
Default:AthDefaultPhysStatusFormat
"%t (%o/%u) | %c | %n | %l | %m | %0 | %1 | %2 | %3 | %4 | %5 | %6 | %7 | %8 | %9 <%h> [%f]"
Context:server config
Status:Extension
Module:mod_athena

This is the format used for requests for physical status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.

top

AthDefaultFarmStatusFormat

Description:
Syntax:AthDefaultFarmStatusFormat format string
Default:AthDefaultFarmStatusFormat
"%n: (%o) algo: %a %h expect update: %e (chosen: %c)\n
   down url: %d offline url: %D\n
   members: %m\n"
Context:server config
Status:Extension
Module:mod_athena

This is the format used for requests for farm status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.

top

AthDefaultEngineStatusFormat

Description:
Syntax:AthDefaultEngineStatusFormat format string
Default:AthDefaultEngineStatusFormat
"path: %P down url: %d farms: %f physicals: %p nominate per update (0/1): %n\n
defaults for farms:\n
   algorithm: %a %h\n
   expect update: %e\n
   ttl: %t"
Context:server config
Status:Extension
Module:mod_athena

This is the format used for requests for engine status, when no format is specified. See Runtime management for format codes. In the conf file, these should be single line strings.