Commentaires (0)

Par mc, 12 octobre 2009 12 h 47 min

tips of this post are cool even if you don’t deal with iso2709.

i wrote a post to explain how sweet it can be to use vim to deal with stdout. Now i want to see the content of a iso2709 file.

as the record separator of iso2709 is « \x1d » but vim

- can’t define a separator different from the standard ones ( combining « \r » and « \n » )
- is very bad to manage very long lines (what an iso2709 file actually is for it)

to navigate in your file, it would be cool to append a « \n » just after « \1xe » (field separator) but with no side effect on the original file. Perl do it easily by mixing -E -0 and -n.

-e flag is to execute perl code ( -E to include perl 5.10 features )

perl -E 'say "hello"'

-n make this code executed for each lines of a read file ( $_ : line content, $. : line number )

cat /etc/hosts
# can be written
perl -nEprint /etc/hosts

grep -n localhost /etc/hosts
# can be written
perl -nE '/localhost/ and print "$. : $_"' /etc/hosts

-0 can change the record separator so

perl -0x1e -nEsay biblio.iso2709

read the file field by field ( -0×1xe ) and append a « \n » (say does it) at the end of each one. Pipe it to vi and you’ll have a fast way to walk through your file.

perl -0x1e -nEsay biblio.iso2709  | vim -

also, set dy (display) to uhex to see the hex codes of the separators ( « \x1d » for the record, « \x1e » for field, « \x1f » for subfield)

:set dy=uhex

also note that perl -n accepts some sed style range

# just see the records 10 to 23
perl -0x1d -nE'  10..23 and print' biblio.iso2709  | vim -

# just see from record 10 to the record that contains the word 'plan9'
perl -0x1d -nE'  10../plan9/ and print' biblio.iso2709  | vim -

and the awk style BEGIN and END blocks

# count the number of records
perl -0x1d -nE 'END { say $. }' biblio.iso2709

Commentaires (0)

Par mc, 26 août 2009 15 h 36 min

when you create a new js file, you put it in the

koha-tmpl/WHERE-tmpl/prog/en/js/foo.js
# WHERE can be intranet or opac

now, in your template:

<script type="text/javascript" src="koha-tmpl/WHERE-tmpl/prog/en/js/foo.js"></script>

will fail if you use a translation. so write:

<script type="text/javascript" src="<!-- TMPL_VAR NAME='themelang' -->/js/behaviors.js"></script>

themelang is added in template C4::Output::gettemplate function (as you can easily see in the documentation)

Commentaires (0)

Par mc, 18 août 2009 10 h 38 min

I used and read a lot of tools since i’m at biblibre. One thing that surprised me is that lot of them completly ignore the unix philosophy. Most of them tries to make too many things in a monolithic code and aren’t usable as filter (my first concern is about the koha bulkmarcimport.pl).

I wonder if this philosophy get lost or has ever existed in the librarian culture? please feedback.

Commentaires (0)

Par mc, 18 août 2009 9 h 51 min

hdl sent this link to me: a tool that make basic translations between marc fields (as copy, move, …). It reminds me my attempt to write a marcawk in which i tied MARC records in hashes.

the ed summer’s tool is very easy to use:

subjmap --translate=650ab

in marawk:

marawk -each '
    @{$_{650}{qw<a b>}} = @{$_{650}{qw<b a>}};
    print $_->as_usmarc;
'

in counterpart: my tool is more powerfull. The ultimate goal was to write
those kind of code to generate html reports

marawk -MCGI=:standard \
    -begin 'print start_html("book list"), start_table;'
    -each 'print Tr( map td($_)
	, $_{210}{a} . $_{210}{b}
	, $_{200}{a}
    )'
    -end 'print end_table, end_html;'
 
'

everything works but the automatic tie of subfields as hash
(i just had no time to investigate). It can be worked around easily with my
recent MARC::Mapper: it’s easy to patch marawk to write:

marawk -as unimarc -MCGI=:standard \
    -begin 'print start_html("book list"), start_table;'
    -each 'print Tr( map td($_)
	, $_->title 
	, $_->author
    )'
    -end 'print end_table, end_html;'

Commentaires (0)

Par mc, 6 août 2009 22 h 13 min

I spent this week migrating the legacy customer data from an old ILS to koha. This is a very funny job because you deal with a lot of data that interact each others. It’s not a collaborative work so you’re on your own to do this.

But there is a dumb part: the MARC format (a format used by libraries since decades for data exchanges). In MARC format, you can’t write

my @barcodes = $self->barcode;

but

map { $_->subfield('h') } $self->field('995');

You may think that it would be easy create an abstraction layer

sub barcodes { 
    map { $_->subfield('h') } (shift)->field('995');
}

But guess what? ‘h’ and ‘995′ depends to the standard you’re dealing with (USMARC,UNIMARC,…) and the local conventions of your ILS. So noone wrote an useless abstraction library. As data migrations are one shots code, it’s borring to write a complete abstraction for your specific case.

Hey! perl is a dynamic langage so it can write it for me: giving a field and a subfield, i can write closures:

sub create_closure { 
    my ($field,$subfield) = @_;
    sub { map { $_->subfield($subfield) } (shift)->field($field); }
};
 
my $get_barecodes = create_closure(qw< 995 a >);
say for $get_barecodes($record);

now i want this closure available as a function of your namespace. rgs++ gimme the solution:

*barcodes = $closure;
# or 
my $function_name = 'barcodes';
my $function_name_in_foo = 'Foo::barcodes';
*{$function_name} = $closure;
*{$function_name_in_foo} = $closure;

so now, what i need is just a hash that contains the name of the functions as keys and the field/subfields as values.

our %map = (
    item_barcode => [qw<995 h>]
    , isbn       => [qw<200 a>]
);

this is my complete and usable code. I’ll use it tomorrow to make my code easier to debug (yeah … still have bugs on my migration) but i still now there are tons of usages possible behind it.

#! /usr/bin/perl
use 5.10.0;
use utf8;
use strict;
use warnings;
use YAML;
 
package MARC::Mapper;
no strict;
 
sub create {
    my ($map) = @_;
    while( my ($function_name, $mapping) = each %$map ) {
	my $closure = sub {
	    my ($self) = shift || $_;
	    my $ref = ref $self;
	    if ($ref eq 'MARC::Field') {
		$self->subfield($$mapping[1])
	    } elsif ($ref eq 'MARC::Record') {
		map $_->subfield(
		    $$mapping[1]), $self->field($$mapping[0]
		);
	    }
	};
	*{$function_name} = $closure;
	my $export = (caller)[0]."::$function_name";
	*{$export} = $closure;
    }
}
 
use strict;
package main; 
use Test::More 'no_plan';
use MARC::Record;
 
my $r =  MARC::Record->new;
$r->append_fields( MARC::Field->new(995,('')x2, h => 'pouet'  ));
 
our %map = (
    item_barcode => [qw<995 h>]
);
 
my $expected = 'pouet';
MARC::Mapper::create(\%map);
 
is( $expected, (MARC::Mapper::item_barcode($r))[0], 'from MAPPER with record'); 
is( $expected, (item_barcode($r))[0], 'from main with record');
$_ = $r;
is( $expected, (item_barcode())[0], 'with $_ and record');
 
my $f = $r->field('995');
 
is( $expected, (MARC::Mapper::item_barcode($f))[0], 'from MAPPER with field'); 
is( $expected, (item_barcode($f))[0], 'from main with field');
$_ = $f;
is( $expected, (item_barcode())[0], 'with $_ and field');

Commentaires (0)

Par mc, 27 juin 2009 18 h 51 min

Using a terminal based editor is cool: you can remotely edit the code on our server and share your screen to make a collaborative work session with someone elsewhere (screen -xS koha). But some claims that they work faster with IDEs. I want to share my tips for koha editing.

[color=blue]always from the koha root[/color]

When i start a koha devel session, i change directory to koha root and update the code and refresh the tags

cd ~/src/koha
git pull --rebase
ctags --languages=perl -R -f TAGS

[color=blue]tags[/color]

If you don’t know about tags: this is an amazing way to navigate into the code:

- goto a function call (cursor under the function name)
- split the window ( <c-w>f )
- call the tag ( <c-]> )

you have now 2 windows: one on the function call, one on the function def.
you can go to any other tag: type :tag Add<tab> to see all koha functions begining by tags.

tags can also be used to complete the code: in insertmode, type Add<c-x]> and choose the function name in the menu.

[color=blue]open templates[/color]

this is my ~/.vim/plugin/koha. it provides easy way to access to a template or include:

- drive the cursor over the name of the template
- in normal mode, type ,to (template open).
- ensure it’s the good template name
- type enter
- et voila …

" if vim is in a koha root
if isdirectory('C4')
	let g:koha_itmpl='koha-tmpl/intranet-tmpl/prog/en/modules' " intranet templates
	let g:koha_iinc='koha-tmpl/intranet-tmpl/prog/en/includes' " intranet includes
 
	" ,te: template edit: just prepare :e with correct path to template root 
	" ,to; template open: just prepare :e with correct path to the template
	" under the cursor
	" ,ie and ,io are the same for includes 
	nnoremap ,te :e <c-r>=g:koha_itmpl<cr>
	nmap ,to ,te/<c-r><c-f>
	nnoremap ,ie :e <c-r>=g:koha_iinc<cr>
	nmap ,io ,ie/<c-r><c-f>
 
	" add include in the path ... so gf works inside templates
	exec 'set path+='.g:koha_iinc
endif

[color=blue]edit templates[/color]

this is my ~/.vim/plugin/html_template.vim

inoremap <V <!-- TMPL_VAR NAME="" --><c-o>F"
inoremap <L <!-- TMPL_LOOP NAME="" --><!--/TMPL_LOOP --><c-o>F"
inoremap <I <!-- TMPL_IF X><!-- /TMPL_IF --><esc>FXs
inoremap <U <!-- TMPL_UNLESS X><!-- /TMPL_UNLESS --><esc>FXs
inoremap <EI <!-- TMPL_ELSIF NAME="" --><c-o>F"
inoremap <EL <!-- TMPL_ELSE -->
inoremap <# <!-- TMPL_INCLUDE NAME="" --><c-o>F"

now, in insert mode: type <V or <I or ..

et voila

HTH

Commentaires (0)

Par mc, 14 mai 2009 9 h 34 min

Ce matin sur biblibre … pas le temps de lire alors je note …

john:

10 commandments
white sockets

nahuel:

opengoo (peut-etre pour biblibre?).

Il dit aussi qu’il faut promouvoir des systemes de vente comme spotify est une bonne chose (moi ca me fait penser a la licence globale sans la régul.).

nicomo

SKOS, RDF, Linked data.

Commentaires (0)

Par mc, 27 avril 2009 11 h 28 min

The adoption of Carp was evoked during the last IRC meeting (looong time ago) but it is not used for the moment. So i want to share 2 tips with you:

First, Concider using Devel::SimpleTrace makes warn and die behave as Carp functions (using traces).

Second, koha have a weak error management (historical issue) and it can be very usefull for you developpers to have a warn that acts as a die (just during the developpement sessions). It can be done very easily by redefining the warn signal:

# warn now act as a die 
$SIG{__WARN__} = sub { die @_ };
 
# so you can catch it!
eval { warn 'mispellers of the world, UNTIE' }; 
$@ and say qq(caught "$@");

Commentaires (0)

Par mc, 23 avril 2009 18 h 07 min

I regulary recieved enthusiastics emails since a wrote cpan2port and
successfully installed koha on macosx. Today, as the answer seems to be almost
identical most of the time, i blog it:

I wrote cpan2port to install koha and i was able to create every required
portfiles for it. I didn’t put them in the port tree because i knew that i
would not be enable to maintain them. Unfortunately, i have no more time to
work on it (if you want me to contribute again, concider funding
Biblibre, my employer).

Ryan Schmidt put cpan2port in contrib and now maintain it.

So how to feed your local repository to install koha?

1) Extract the dependancies list from the sources:

perl -lne "/PREREQ_PM =>/../}/ and /'(.*?)'/ and print \$1" Makefile.PL > DEPS

2) delete from DEPS where the Portfile still exist or the module is a part of
the core distribution.

3) feed your repo

mkdir repo && 
    cd repo && 
    cpan2port -t < ../DEPS

FAQ:

AFAIR: « Test::Harness is not perl porter » means that Test::Harness is a
package from core distribution so you don’t have to package it (so it’s not a good idea to package it again) but my script don’t deal with dual life modules
such as Test::Harness. I don’t remember if there is a way to deal with
it. BTW: i didn’t create a Portfile for it and my koha works fine.

Commentaires (0)

Par mc, 11 avril 2009 0 h 52 min

thanks for French PM, i looked at SQL::Abstract in order to replace my SQHelper. The fact is that SQL Abstract force to rebuild the statement at every time. SQHelper does not!

Data::Phrasebook was also mentionned and this is closer to the things i want. but i tasted my map_hash and i can’t do without it yet