Commit f9a3dff8 authored by Eric German's avatar Eric German

initial import : implementes replication and soap service

git-svn-id: svn://svn.forge.objectweb.org/svnroot/lemonldap/trunk@44 1dbb9719-a921-0410-b57f-c3a383c2c641
parent b0b25ca5
Revision history for Perl extension Apache::Session::Memorycached.
1.0 Sun Oct 3 15:05:46 2004
- original version; created by h2xs 1.22 with options
-XAn Apache::Session::Memorycached
1.1 Oct 8 2004
- Add 'timeout' session parameter
2.0.0 May 04 2006
- optimize persistant connection
- add memd.pl in scripts directory
2.1.0 Nov 17 2006
- add reverse reference on principal identity
2.2.0 Feb 09 2007
- add two replication daemons
- add SOAP service
Changes
lib/Apache/Session/Lock/Memorycached.pm
lib/Apache/Session/Memorycached.pm
lib/Apache/Session/MemcachedClient.pm
lib/Apache/Session/MemcachedReplicator.pm
lib/Apache/Session/Store/Memorycached.pm
Makefile.PL
MANIFEST This list of files
META.yml
README
scripts/memd.pl
scripts/client_memcached.pl
scripts/slurp_memcached.pl
scripts/MemcachedSOAPClass.pm
scripts/MemcachedSOAP.cgi
scripts/statTest.pl
scripts/getTest.pl
scripts/setTest.pl
t/1.t
t/test-apache-session.t
# http://module-build.sourceforge.net/META-spec.html
#XXXXXXX This is a prototype!!! It will change in the future!!! XXXXX#
name: Apache-Session-Memorycached
version: 2.2.0
version_from: lib/Apache/Session/Memorycached.pm
installdirs: site
requires:
Apache::Session: 0.0
Cache::Memcached: 1.0.12
distribution_type: module
generated_by: ExtUtils::MakeMaker version 6.17
use ExtUtils::MakeMaker;
# See lib/ExtUtils/MakeMaker.pm for details of how to influence
# the contents of the Makefile that is written.
WriteMakefile(
NAME => 'Apache::Session::Memorycached',
VERSION_FROM => 'lib/Apache/Session/Memorycached.pm', # finds $VERSION
PREREQ_PM => {Apache::Session=> '0.0',
Cache::Memcached=> '1.0.12',
}, # e.g., Module::Name => 1.1
($] >= 5.005 ? ## Add these new keywords supported since 5.005
(AUTHOR => 'Eric German <germanlinux@yahoo.fr>') : ()),
);
NAME
Apache::Session::Memorycached - An implementation of Apache::Session
SYNOPSIS
use Apache::Session::Memorycached;
tie %session, 'Apache::Session::Memorycached', $cookie, {
'servers' => ["10.75.1.19:11211"], #all write operations
'local' => ["localhost:11211"], #read-only operations
'timeout' => '300'
};
tie %s, 'Apache::Session::Memorycached', undef,
{servers => ['mymemcachedserver:port'],
'timeout' => '300',
'updateOnly' => 1 ,
'principal' => uid,
};
In order to optimize the network ,you can use a local memcached server.
All read-only opération are sending fisrt at local server .If you need
write ou rewrite data , the data is sending at the principal memcached
sever and local cache too for synchronisation.
note : 'updateOnly' => 1 just realize up-date operation not init
operation. Init operation is use in order to book and lock the number
session but it's not available in this module
'principal' => uid : this parameter is use to create reverse reference
like this : MD5_hex(uid) => id_session in memcached server . By this it usefull to retrieve id_session from principal name . And add uid_MD5 => MD5_hex(uid) in main session .
DESCRIPTION
This module is an implementation of Apache::Session. It uses the
memcached system backing store . You may specify servers (principal) and
locals caches for locking in arguments to the constructor. See the
example, and the documentation for Apache::Session::Store::Memorycached
and Cache::Memcached .
REPLICATION
Now Apache::Session::Memorycahed inclues replication between memecached servers
Two new components provide a replication service .
First package is Apache::Session::MemcachedReplicator
Second is Apache::Session::MemcachedClient
It's now possible to do replication master to slave or master to master
see man pages and scripts .
SOAP service
Now Apache::Session::Memorycached inclues a SOAP service in order to set or
get %session in any language . The SOAP service translates data in Perl hashes
Installation of SOAP service
All scripts are in scripts directory
Put MemcachedSOAPClass.pm and MemcachedSOAP.cgi in the cgi-bin directory of your apache server with the appropriate right (x) .
Change in MemcachedSOAP.cgi the memcached server address .
(line 11 : $machine = 'ip.ip.ip.ip:11211'; )
Try the three scripts statTest.pl (first !) then getTest.pl finish with setTest.pl.
The lemonldap project (SSO under GPL) uses this module
AUTHOR
This module was written by eric german <germanlinux@yahoo.fr>.
Completed by Habib ZITOUNI <zitouni.habib@gmail.com> and
Hamza AISSAT<asthamza@hotmail.fr>
SOAP service is a contribution of Casimir ANTUNES .
SEE ALSO
Apache::Session::DB_File, Apache::Session::Flex,
Apache::Session::MemcachedClient,Apache::Session::MemcachedReplicator,
Apache::Session::MySQL, Apache::Session::Postgres, Apache::Session
############################################################################
#
# Apache::Session::Lock::Memorycached
# Copyright(c) eric german <germanlinux@yahoo.fr>
# Distribute under the Artistic License
#
############################################################################
package Apache::Session::Lock::Memorycached;
use strict;
use vars qw($VERSION);
$VERSION = '1.0';
sub new {
my $class = shift;
return bless { read => 0, write => 0, opened => 0, id => 0 }, $class;
}
sub acquire_read_lock {
my $self = shift;
my $session = shift;
return if $self->{read};
if (!$self->{opened}) {
$self->{opened} = 1;
}
$self->{read} = 1;
}
sub acquire_write_lock {
my $self = shift;
my $session = shift;
return if $self->{write};
if (!$self->{opened}) {
$self->{opened} = 1;
}
$self->{write} = 1;
}
sub release_read_lock {
my $self = shift;
my $session = shift;
die unless $self->{read};
if (!$self->{write}) {
$self->{opened} = 0;
}
$self->{read} = 0;
}
sub release_write_lock {
my $self = shift;
my $session = shift;
die unless $self->{write};
if ($self->{read}) {
}
else {
$self->{opened} = 0;
}
$self->{write} = 0;
}
sub release_all_locks {
my $self = shift;
my $session = shift;
if ($self->{opened}) {
}
$self->{opened} = 0;
$self->{read} = 0;
$self->{write} = 0;
}
sub DESTROY {
my $self = shift;
$self->release_all_locks;
}
sub clean {
my $self = shift;
my $dir = shift;
my $time = shift;
}
1;
=pod
=head1 NAME
Apache::Session::Lock::File - Provides mutual exclusion using flock
=head1 SYNOPSIS
use Apache::Session::Lock::File;
my $locker = new Apache::Session::Lock::File;
$locker->acquire_read_lock($ref);
$locker->acquire_write_lock($ref);
$locker->release_read_lock($ref);
$locker->release_write_lock($ref);
$locker->release_all_locks($ref);
$locker->clean($dir, $age);
=head1 DESCRIPTION
Apache::Session::Lock::Memorycached fulfills the locking interface of
Apache::Session. NO locking is using .
=head1 CONFIGURATION
none
=head1 NOTES
=head1 AUTHOR
This module was written by eric german <germanlinux@yahoo.fr>
=head1 SEE ALSO
L<Apache::Session>
package Apache::Session::MemcachedClient;
use Apache::Session::Memorycached;
use Fcntl qw(:DEFAULT :flock);
$| = 1;
our $VERSION = '2.1.1';
sub new {
my $class = shift;
my %args = @_;
my $self;
$self = \%args;
bless $self, $class;
}
sub run {
my $self = shift;
my $file_input = $self->{in_file};
my $file_output = $self->{out_file};
my $naptime = $self->{naptime} || '1';
my $safetime = $self->{safetime} || '900'; # 15 minutes
for ( ; ; ) {
sysopen( FH, $file_input, O_RDWR | O_CREAT ) || die "$file_input $!\n";
flock( FH, LOCK_EX );
my @ligne = <FH>;
seek( FH, 0, 0 );
truncate( FH, 0 );
close(FH);
my @log;
for (@ligne) {
( my $session, my $time ) = /^(\w+)\s(\d+)/;
### retrieve session
my $param = $self->{localmemcached};
my $sign = $self->{signature} || 'master';
my %localsession;
my %remotesession;
tie %localsession, 'Apache::Session::Memorycached', $session,
$param;
if ( !%localsession ) {
### error in retrieve session
untie %localsession;
push @log, "FATAL :$session FAILED \n";
next;
}
my %_localsession = %localsession;
untie %localsession;
#######################################
## avoid loop in master2master case ##
#######################################
if ( $_localsession{$sign} ) {
### pehap already replicated
# exept in this case
my $time_origine = $_localsession{$sign};
$time_origine =~ s/#.+$//;
if ( ( time - $time_origine ) < $safetime ) {
push @log, "INFO :$session SYN OK\n";
next;
}
}
$_localsession{$sign} = time . "#" . $time;
#### and send this to the other server
my $paramdist = $self->{remotememcached};
##### ne marche pas $session exist ########
my %remotesession;
tie %remotesession, 'Apache::Session::Memorycached', $session,
$paramdist;
if (%remotesession) {
### error in retrieve remote session
my $time_origine = $remotesession{$sign};
$time_origine =~ s/#.+$//;
next if ( ( time - $time_origine ) < $safetime );
}
%remotesession = %_localsession;
untie %remotesession;
push @log, "INFO :$session REPLICATED\n";
if ( $self->{safety_mode} ) {
my %remotesession;
### we retrieve session from the other memcached server
my $paramdist = $self->{remotememcached};
tie %remotesession, 'Apache::Session::Memorycached', $session,
$paramdist;
my %_remotesession = %remotesession;
untie %remotesession;
if ( $_remotesession{$sign} ) {
push @log, "INFO :$session VERIFIED\n";
}
else {
push @log, "FATAL :$session REPLICATION ERROR\n";
}
}
}
sysopen( FS, $file_output, O_WRONLY | O_APPEND | O_CREAT )
|| die "$file_output $!\n";
flock( FS, LOCK_EX );
for (@log) {
print FS $_;
}
close(FS);
sleep $naptime;
}
}
1;
=pod
=head1 NAME
Apache::Session::MemcachedClient - A component of memcached's replication
=head1 SYNOPSIS
use Apache::Session::MemcachedClient ;
my $rep = MemcachedClient->new(in_file =>"/tmp/logmem1",
out_file =>"/tmp/log1",
naptime => 2 ,
localmemcached => {'servers' => ['localhost:11211'], },
remotememcached =>{'servers' => ['localhost:11311'], },
signature => 'master11211',
safety_mode =>'actived' ,
);
$rep->run ;
exit;
=head1 DESCRIPTION
This module is an implementation of replication for memcached backend session storage . It replicates session created by Apache::Session::Memorycached between master to slave OR master to master.
In input , it reads a file issued from Apache::Session::MemcachedReplicator then it sends session on the other memcached server .
The lemonldap project (SSO under GPL) uses this module
=head1 Options
- in_file : input file .
- out_file : log in output file
- naptime : time between 2 cycles (in second)
- localmemcached : you local server
- remotememcached : you remote server (pehap the slave)
- signature : string used in order to avoid loops replication
- safety_mode : thrue : read on remote server after write in order to be sure of success of replication
see client_memcached.pl in script directory.
=head1 AUTHOR
This module was written by eric german <germanlinux@yahoo.fr>.
=head1 SEE ALSO
L<Apache::Session::MemcachedReplicator>,
L<Apache::Session::Memorycached>,
package Apache::Session::MemcachedReplicator;
use Fcntl qw(:DEFAULT :flock) ;
our $VERSION = '2.1.1';
$| =1;
sub new {
my $class =shift;
my %args = @_;
my $self;
$self=\%args;;
bless $self,$class;
}
sub run {
my $self =shift;
my $file_input = $self->{in_file} ;
my $file_output = $self->{out_file} ;
my $naptime= $self->{naptime}||'1';
for(;;) {
sysopen (FH,$file_input, O_RDWR|O_CREAT) ||die "$file_input $!\n";
flock (FH,LOCK_EX);
my @ligne = <FH> ;
seek (FH,0,0);
truncate (FH,0);
close(FH) ;
my %un_id;
for (@ligne) {
if (/\sset\s/ ) {
( my $session) = /set\s(\w+)\s/ ;
$un_id{$session} = time ;
}
}
sysopen (FS,$file_output, O_WRONLY|O_APPEND|O_CREAT)||die "$file_output $!\n";
flock (FS,LOCK_EX);
for (keys %un_id) {
print FS "$_ $un_id{$_}\n";
}
close (FS) ;
sleep $naptime;
}
}
1;
=pod
=head1 NAME
Apache::Session::MemcachedReplicator - A component of memcached's replication
=head1 SYNOPSIS
use Apache::Session::MemcachedReplicator ;
my $rep = MemcachedReplicator->new(in_file =>"/tmp/memcachedlog",
out_file =>"/tmp/logmem",
naptime => 2 ,
);
$rep->run ;
exit;
=head1 DESCRIPTION
This module reads log's memcached server and write one line by 'set' command .
In order to force memcached to be verbose you must laugth it like this:
memcached -u root -p 11211 -vv 2> /tmp/logmem1 &
The memcached log file must to be in input of MemcachedClient .
The lemonldap project (SSO under GPL) uses this module.
=head1 Options
- in_file : input file .
- out_file : write in output file
- naptime : time between 2 cycles (in second)
see slurp_memcached.pl in script directory.
=head1 AUTHOR
This module was written by eric german <germanlinux@yahoo.fr>.
=head1 SEE ALSO
L<Apache::Session::MemcachedClient>,
L<Apache::Session::Memorycached>,
#############################################################################
#
# Apache::Session::Memorycached
# Apache persistent user sessions on the network with memcached
# Copyright(c) eric german <germanlinux@yahoo.fr>
# Distribute under the Artistic License
#
############################################################################
package Apache::Session::Memorycached;
use strict;
use vars qw(@ISA $VERSION);
$VERSION = '2.2.0';
@ISA = qw(Apache::Session);
use Apache::Session;
use Apache::Session::Generate::MD5;
use Apache::Session::Lock::Memorycached;
use Apache::Session::Store::Memorycached;
sub populate {
my $self = shift;
$self->{object_store} = new Apache::Session::Store::Memorycached $self;
$self->{lock_manager} = new Apache::Session::Lock::Memorycached $self;
$self->{generate} = \&Apache::Session::Generate::MD5::generate;
$self->{validate} = \&Apache::Session::Generate::MD5::validate;
$self->{serialize} = \&Apache::Session::Memorycached::none;
$self->{unserialize} = \&Apache::Session::Memorycached::none;
return $self;
}
sub none {
my $self = shift;
my $session = shift;
return;
}
sub DESTROY {
my $self = shift;
$self->save;
$self->{object_store}->close;
$self->release_all_locks;
}
1;
=pod
=head1 NAME
Apache::Session::Memorycached - An implementation of Apache::Session
=head1 SYNOPSIS
use Apache::Session::Memorycached;
tie %session, 'Apache::Session::Memorycached', $cookie, {
'servers' => ["10.75.1.19:11211"], #all write operations
'local' => ["localhost:11211"], #read-only operations
'timeout' => '300'
};
tie %s, 'Apache::Session::Memorycached', undef,
{servers => ['mymemcachedserver:port'],
'timeout' => '300',
'updateOnly' => 1 ,
'principal' => uid,
};
In order to optimize the network ,you can use a local memcached server.
All read-only opération are sending fisrt at local server .If you need write ou rewrite data , the data is sending at the principal memcached sever and local cache too for synchronisation.
note : 'updateOnly' => 1 just realize up-date operation not init operation.
Init operation is use in order to book and lock the number session but it's not available in this module
'principal' => uid : this parameter is use to create reverse reference
like this : MD5_hex(uid) => id_session in memcached server . By this it usefull to retrieve id_session from principal name . And add uid_MD5 => MD5_hex(uid) in main session .
=head1 DESCRIPTION
This module is an implementation of Apache::Session. It uses the memcached system backing
store . You may specify servers (principal) and locals caches for locking in arguments to the constructor. See the example, and the documentation for Apache::Session::Store::Memorycached and Cache::Memcached .
=head1 REPLICATION
Now Apache::Session::Memorycahed inclues replication between memecached servers
Two new components provide a replication service .
First package is Apache::Session::MemcachedReplicator
Second is Apache::Session::MemcachedClient
It's now possible to do replication master to slave or master to master
see man pages and scripts .
=head1 SOAP service
Now Apache::Session::Memorycached inclues a SOAP service in order to set or
get %session in any language . The SOAP service translates data in Perl hashes
=head2 Installation of SOAP service
All scripts are in scripts directory
Put MemcachedSOAPClass.pm and MemcachedSOAP.cgi in the cgi-bin directory of your apache server with the appropriate right (x) .
Change in MemcachedSOAP.cgi the memcached server address .
(line 11 : $machine = 'ip.ip.ip.ip:11211'; )
Try the three scripts statTest.pl (first !) then getTest.pl finish with setTest.pl.
The lemonldap project (SSO under GPL) uses this module
=head1 AUTHOR
This module was written by eric german <germanlinux@yahoo.fr>.
Completed by Habib ZITOUNI <zitouni.habib@gmail.com> and