Cache entry expiration?

[John Moore-2]

I've implemented Cache and CacheManager using Redis. Am I right in
thinking that Shiro handles all issues to do with cache expiration and
that I shouldn't automatically expire the cache entries in Redis myself?
I deduced this from looking at the EHCache implementation. Please
correct me if I'm wrong. I'll probably still expire the entries after
say 24 hours, to avoid clogging up the db with orphaned items, unless
there's a compelling reason not to do this.

[Les Hazlewood-2]

Shiro will delegate to a cache for persistence of authc, authz and
session data.  The Cache/CacheManager implementation is responsible
for evicting entries based on implementation-specific
time-to-live/time-to-idle or memory constraint settings.

If you're using native sessions and the EnterpriseCacheSessionDAO
(where it is assumed the Cache itself has the ability to overflow to
disk if all available cache memory becomes full), you will want to
ensure that the session cache region does not evict entries at all
(the assumption is that they'd be removed permanently).  When Shiro's
session validation mechanism executes, it will explicitly remove
sessions that are stale or expired.  All other cache regions (authc,
authz data) can be evicted as you see fit.

Another approach for a sessions specifically is to turn off Shiro's
session validation entirely and let the Cache implementation's TTL
setting be the session timeout.  That is, if you set the TTL to, say,
30 minutes, then the Cache eviction mechanism _is_ the session cleanup
mechanism.  This might be more efficient than running a validation job
every N minutes, especially if you have a cluster of web application
instances (i.e. does each web app instance perform validation for the
entire cluster?  do they all do it?).

The key here is that _something_ is responsible for evicting stale or
expired sessions to ensure your session data store does not fill up
unnecessarily.  If you turn off Shiro's built-in validation mechanism,
something else must be responsible for it.

HTH,

Les

On Wed, Feb 8, 2012 at 6:36 AM, John Moore <[hidden email]> wrote:
> I've implemented Cache and CacheManager using Redis. Am I right in thinking
> that Shiro handles all issues to do with cache expiration and that I
> shouldn't automatically expire the cache entries in Redis myself? I deduced
> this from looking at the EHCache implementation. Please correct me if I'm
> wrong. I'll probably still expire the entries after say 24 hours, to avoid
> clogging up the db with orphaned items, unless there's a compelling reason
> not to do this.

[John Moore-2]

On 08/02/12 17:32, Les Hazlewood wrote:
> Another approach for a sessions specifically is to turn off Shiro's
> session validation entirely and let the Cache implementation's TTL
> setting be the session timeout.  That is, if you set the TTL to, say,
> 30 minutes, then the Cache eviction mechanism _is_ the session cleanup
> mechanism.  This might be more efficient than running a validation job
> every N minutes, especially if you have a cluster of web application
> instances (i.e. does each web app instance perform validation for the
> entire cluster?  do they all do it?).

This certainly seems a sensible approach. So how do I turn off Shiro
session validation?

John

[Les Hazlewood-2]

http://shiro.apache.org/session-management.html#SessionManagement-DisablingSessionValidation

Best,

Les

On Wed, Feb 8, 2012 at 9:52 AM, John Moore <[hidden email]> wrote:

> On 08/02/12 17:32, Les Hazlewood wrote:
>>
>> Another approach for a sessions specifically is to turn off Shiro's
>> session validation entirely and let the Cache implementation's TTL
>> setting be the session timeout.  That is, if you set the TTL to, say,
>> 30 minutes, then the Cache eviction mechanism _is_ the session cleanup
>> mechanism.  This might be more efficient than running a validation job
>> every N minutes, especially if you have a cluster of web application
>> instances (i.e. does each web app instance perform validation for the
>> entire cluster?  do they all do it?).
>
>
> This certainly seems a sensible approach. So how do I turn off Shiro session
> validation?
>
> John

[John Moore-2]

On 08/02/12 20:15, Les Hazlewood wrote:
> http://shiro.apache.org/session-management.html#SessionManagement-DisablingSessionValidation
>
>
Great, thanks!

[Les Hazlewood-2]

> def sessionValidationScheduler = new org.apache.shiro.session.mgt.ExecutorServiceSessionValidationScheduler()
> sessionValidationScheduler.setInterval(30000)
> shiroSecurityManager.sessionManager.sessionValidationScheduler = sessionValidationScheduler
>
> One thing worthy of note is that unless I have the last 3 lines, where I
> explicitly configure a sessionValidationScheduler, nothing is written to
> the log, presumably meaning that there is no default validation
> scheduler. I don't know whether this is expected, or whether it's a
> quirk of the plugin.

Ah, I see what is happening.  I think this is a bug - or at least
confusing at best.  Let me explain:

When a SessionValidationScheduler instance is created outside of
Shiro's internal mechanisms and explicitly set on the sessionManager
(an AbstractValidatingSessionManager instance), the sessionManager
will only ensure that scheduler.enableSessionValidation(); is invoked.
 It does _not_ ensure that the scheduler is configured or even that it
references the sessionManager.  It is assumed that whoever created the
Scheduler instance did this explicitly.

So in the code snippet above, the
ExecutorServiceSessionValidationScheduler instance is created, but it
is not given a reference to the SessionManager.  A scheduler's primary
purpose is to periodically invoke the
ValidatingSessionManager.validateSessions() method.  Without a
reference to the sessionManager, it cannot function.  So in the above
code, while the scheduler is created, and it might start running, it
never actually validates any sessions.

I agree this is not very 'Shiroy' - you would assume that when setting
a scheduler, the sessionManager should probably automatically make
itself the scheduler's target.

So, there are two solutions for this immediate problem:

1.  Don't configure the Scheduler explicitly and let Shiro create it.
You can set the validation interval as a property on the
sessionManager instance and it will be relayed to the implicitly
created Scheduler.  For example:

securityManager.sessionManager.sessionValidationInterval = timeInMillis

2.  Create any Scheduler instance you wish, but ensure it has a
reference to the sessionManager:

sessionValidationScheduler =
org.apache.shiro.session.mgt.ExecutorServiceSessionValidationScheduler
sessionValidationScheduler.interval = timeInMillis
sessionValidationScheduler.sessionManager = $sessionManager
securityManager.sessionManager.sessionValidationScheduler =
$sessionValidationScheduler

Either should do it.

Now that that's covered, I think I should make some points about
validation scheduling in a clustered environment:

*Clustering*

In a clustered environment where you might have N number of
application nodes, all of which are Shiro enabled - and all of which
point to the same session store - you only really need one of those
app instances at any point in time to perform session validation.  It
just doesn't make sense to run a scheduler on all nodes if only one
node needs to be doing the work.  Running the validation only one node
also means the other nodes are freed up for other things (like
processing requests).

As such, you ideally would not want to use the
ExecutorServiceSessionValidationScheduler since it is not 'cluster
aware'.  One option is to have only one node have a slightly different
Shiro config so it only did the session validation, but this is
brittle IMO - you have to maintain separate configs, and if that node
died, another wouldn't automatically take its place, etc.

So there are two options for session validation in clusters that I
think works quite well:

1.  Turn off Shiro's session validation mechanism entirely and rely on
your clustered cache to evict entries.  The cache entry TTL is
effectively the session timeout.  I believe you're already going down
this route, and I think it is a sound option.  Assuming you don't need
to retain stopped/expired sessions for reporting purposes, this is
also the easiest approach (on the Shiro side at least) because it
reduces your Shiro config.

2.  Use a cluster-compatible SessionValidationScheduler
implementation.  The Shiro quartz module exists explicitly because of
this need:  Quartz has the ability to obtain a cluster-wide lock (e.g.
using an RDBMS table for example) to ensure that one node and only one
node at a time runs the Quartz Job.  This ensures that other nodes are
free to process requests instead of all nodes attempting to validate
active sessions at around the same time (which would be extremely
inefficient and potentially cause performance problems in a cluster).

Well, I hope that helps clear things up!  Let us know how it goes...

Best,

--
Les Hazlewood
CTO, Katasoft | http://www.katasoft.com | 888.391.5282
twitter: @lhazlewood | http://twitter.com/lhazlewood
katasoft blog: http://www.katasoft.com/blogs/lhazlewood
personal blog: http://leshazlewood.com