IPhone – Corruptions within Calendar: duplicates. How to analyse their cause and fix

I haven't yet a correct explanation of these corruptions, but at least I wrote a solution to have a clear vision of the extent of the damage and to fix it.

Here is a perl script: duplicate.pl:

$ cat <<'eof' >duplicate.pl
#!/usr/bin/perl
use strict ;
use warnings ;

# program reading on its standard input a file under
# ics format exported by iCal or Calendar
# Both only export one calendar at a time

# file = name of created file for a given calendar

my %file = () ;

# filedesc = file descriptor of the created calendar file

my %filedesc = () ;

# hash of all unduplicated events local_event_id
my %events = () ;

# current event storage
my @event = () ;
my $dtstart = '' ;
my $dtend = '' ;

# number of events analysed
my $num_event = 0 ;
my $duplicate = 0 ;
my $calendar = '' ;

# state booleans
my $in_header = 1 ;
my $in_event = 0 ;
my $in_summary = 0 ;
my $line = '' ;
my $summary = '' ;

# local event identifier :      summary;dtstart;dtend
# because ';' is never used within a name
my $local_event_id = '' ;

while (<STDIN>) {
        $line = $_ ;

# header :      BEGIN:VCALENDAR
#               ...
#               BEGIN:VEVENT

        if ( $in_summary ) {

#               continuation line of summary

                if ( $line =~ /^ (.+)\r\n$/ ) {
                        $summary .= $1 ;
                } else {

#                       end of summary continuation lines analysis

                        $in_summary = 0 ;
                }
        }
        if ( $line =~ /^SUMMARY[^:]*:(.+)\r\n$/ ) {
                $summary = $1 ;
                $in_summary = 1 ;
        } elsif ( $line =~ /^BEGIN:VEVENT/) {
                if ( $in_header ) {
                        $in_header = 0 ;

#                       print every lines of event or header

                        foreach $line (@event) {
                                printf {$filedesc{$calendar}} "%s", $line ;
                        }
                }
                $in_event = 1 ;
                @event = () ;
        } elsif ( $line =~ /^X-WR-CALNAME:(.+)\r\n$/) {
                $calendar = $1 ;

#               create .ics file

                if ( ! defined $file{$calendar} ) {
                        $file{$calendar} = $calendar . ".ics" ;
                        if ( -e $file{$calendar} ) {
                                die "$file{$calendar} already exists\n" ;
                        }
                        open ($filedesc{$calendar}, ">", $file{$calendar}) ;

#                       print every lines of header

                        foreach $line (@event) {
                                printf {$filedesc{$calendar}} "%s", $line ;
                        }
                        @event = () ;
                        $in_event = 0 ;
                }
#               printf STDOUT "calendar = %s\n", $calendar ;
#               printf STDOUT "file = %s\n", $file{$calendar} ;
#               printf STDOUT "fh = %d\n", $filedesc{$calendar} ;

        } elsif ( $line =~ /^DTSTART[^:]*:(.*)\r\n$/ ) {
                $dtstart = $1 ;

#               printf STDOUT "DTSTART = %32s\n", $dtstart ;

        } elsif ( $line =~ /^DTEND[^:]*:(.*)\r\n$/ ) {
                $dtend = $1 ;

#               printf STDOUT "DTEND = %32s\n", $dtend ;

        } elsif ( $line =~ /^END:VEVENT/ ) {

# it's only on closing an event definition that we have
# a complete local event identifier

                $local_event_id = "$summary" . ";" . "$dtstart" . ";" . "$dtend" ;


                if ( defined ( $events{"$local_event_id"} )) {

#                       duplicate event

                        printf STDOUT "\n\tduplicate\t%s\n", $local_event_id ;

#                       free event storage

                        @event = () ;
                        $in_event = 0 ;
                        $duplicate++ ;
                } else {

#                       new event
                        $events{$local_event_id} = 1;

                        if ($in_event) {

#                               print every lines of stored event

                                foreach $line (@event) {
                                        printf {$filedesc{$calendar}} "%s", $line ;
                                }
                                @event = () ;
                                $in_event = 0 ;
                                $num_event++ ;
                                $in_header = 1 ;

#                               show progress
                                if (($num_event % 100) == 0) {
                                        printf STDOUT "\n%8d", $num_event ;
                                } else {
                                        print STDOUT "." ;
                                }
                        }
                }
        } elsif ( $in_event == 0 ) {
                $in_header = 1 ;
        }

#       store every line of event or header

        if ($in_event || $in_header) {
                push (@event, $line) ;
        }

}

printf STDOUT "\nevents:%12d\n", $num_event ;
printf STDOUT "duplicates:%8d\n", $duplicate ;

#                       print every lines of ending

                        foreach $line (@event) {
                                printf {$filedesc{$calendar}} "%s", $line ;
                        }

close ($filedesc{$calendar}) ;
exit 0 ;
eof
$ chmod u+x duplicate.pl

And here is the way I used it:

1. From Calendar, export a given calendar through:

   File > Export > Export...

   let's say it is: Documents/Calendar/2015/professionnal.ics

   NB.: this will be a backup in case of problem, and a much more practical one than any file manipulation with the help of Time Machine

2. Install the above script as:

   ~/Documents/Calendar/src/duplicate.pl
   

3. Go in directory to test for fixed version, for example:

   mkdir ~/Documents/Calendar/2015.fixed
   cd ~/Documents/Calendar/2015.fixed
   

4. Run duplicate.pl:

   ../src/duplicate.pl <../2015/professionnal.ics
   

   which will display the number of events read and the number of duplicate found and create here a fixed version of the calendar:

   ~/Documents/Calendar/2015.fixed/professionnal.ics
   

5. Compare the result with the original corrupted version to check everything is OK:

    diff ../2015/professionnal.ics .
   

6. Within Calendar select the calendar professionnal and delete it through:

   Edit > Delete

7. Import the fixed one through:

   File > Import > Import...