More work on the kal aka cal.py command. It can now do around 80% of all the stuff calendar-cli can do, plus more.

10 files changed, 731 insertions, 46 deletions

diff --git a/NEW_CLI.md b/NEW_CLI.md
index ea7728e..2a5834c 100644
--- a/NEW_CLI.md
+++ b/NEW_CLI.md
@@ -1,4 +1,4 @@
-I'm currently working on a cal.py aka kal that eventually will replace calendar-cli.py
+I'm currently working on a cal.py aka kal that eventually will replace calendar-cli.py.  This is the long document with design thoughts, and it may not be completely in sync with what I'm actually implementing.  Eventually, read the shorter [USER_GUIDE](USER_GUIDE.md) for a more up-to-date and shorter document.
 
 ## General thoughts
 
diff --git a/README.md b/README.md
index 085a719..5d20eb7 100644
--- a/README.md
+++ b/README.md
@@ -13,6 +13,8 @@ There is a "competing" project at https://github.com/geier/khal - you may want t
 New vs old interface
 --------------------
 
+DO YOU HAVE OPINIONS ON WHAT COLOR TO PAINT THE BIKE SHED WITH?  VISIT https://github.com/tobixen/calendar-cli/issues/88 NOW!
+
 calendar-cli.py is the old interface, it will hang around and be supported for some time to come.  cal.py is the new interface, but until version 1.0 is ready, there will still be functionality in calendar-cli that isn't mirrored to cal.py.
 
 I wanted a short and easy command name, since `cal(1)` is already a popular Unix command, I'm considering to install it into /usr/bin with the name `kal`, but I'm a bit uncertain (seems either like a bad typo or an attempt on localizing the command into my native language?)
diff --git a/USER_GUIDE.md b/USER_GUIDE.md
new file mode 100644
index 0000000..d66c1c9
--- /dev/null
+++ b/USER_GUIDE.md
@@ -0,0 +1,117 @@
+# User guide for kal
+
+This document was updated 2022-10-09.  The kal command is under heavy development, this document may not always be up-to-date.
+
+The new command kal is under development.  It can already do nearly all the things calendar-cli can do.
+
+## Command structure
+
+Commands are on this format:
+
+```bash
+kal --global-options command --command-options subcommand --subcommand-options
+```
+
+The most up-to-date documentation can always be found through `--help`, and it's outside the scope of this document to list all the options.
+
+```bash
+kal --help
+kal command --help
+kal command subcommand --help
+```
+
+## Main commands
+
+* test - verify that it's possible to connect to the server
+* add - for adding things to the calendar(s)
+* select - for selecting, viewing, editing and deleting things from the calendar(s).
+
+## Global options
+
+The global options are for setting the connection parameters to the server and choosing what calendar(s) to operate at.  Connection parameters may be typed in directly:
+
+* `--caldav-*` to set the server connection parameters
+* `--calendar-*` to choose a calendar.  If nothing is specified, the first calendar found will be utilized (on some calendar servers, this will be the default calendar).  It's possible to specify those parameters multiple times.
+
+It's recommended to rather use a config file (though not yet supported as of 2022-10-09).  Those options can be used for specifying a config file:
+
+* `--config-file`
+* `--config-section`
+
+The default (though not yet supported as of 2022-10-09) is to utilize the `default` section under `$HOME/.config/calendar.conf`
+
+Multiple config sections can be specified, which may be useful for selecting things from multiple calendars.
+
+## Adding things to the calendar
+
+Generally it should be done like this:
+
+```
+kal add ical --ical-file=some_calendar_data.ics
+kal add event --event-options 'New years party' '2022-12-31T17:00+8h'
+kal add todo --todo-options 'Prepare for the new years party'
+kal add journal --journal-options "Resume from the new years party" 2022-12-31 "It was awesome.  Lots of delicous food and drinks.  Lots of firework."
+```
+
+(journals not supported yet as of 2022-10-09)
+
+Most often, no options should be given to the command `add` - with the exception if one wants to add things to multiple calendars in one command.
+
+Most of the options given after the subcommand is for populating object properties like location, categories, geo, class, etc.
+
+## Selecting things from the calendar
+
+```
+kal select --selection-parameters select-command
+```
+
+It's usually a good idea to start with the select-command `list`, for instance:
+
+```
+kal select --todo --category computer-work list
+```
+
+Some calendar server implementations require  `--todo` or `--event` to always be given when doing selects, others not.
+
+### Listing objects
+
+Events can either be output as ics, or through a template.
+
+The templating engine is built on top of the python `string.format()`.  To learn the basics of `string.format()`, w3schools have some nice interactive thing on the web, https://www.w3schools.com/python/ref_string_format.asp
+
+Text fields can be accessed directly i.e. like this:
+
+```
+kal select --todo list --template='{SUMMARY} {DESCRIPTION} {LOCATION}'
+```
+
+Dates can be accessed through the dt property, and can be formatted using strftime format, like this:
+
+```
+kal select --event list --template='{DTSTART.dt:%F %H:%M:%S}: {SUMMARY}'
+```
+
+If a property is missing, the default is to insert an empty string - but it's also possible to put a default value like this:
+
+```
+kal select --event list --template='{DTSTART.dt:%F %H:%M:%S}: {SUMMARY:?(no summary given)?}'
+```
+
+It's even possible to make compounded defaults, like this:
+
+```
+kal select --todo list --template='{DUE:?{DTSTART.dt:?(Best effort)?}?:%F %H:%M:%S}: {SUMMARY:?(no summary given)?}'
+```
+
+One thing that may be particularly useful is to take out the UID fields.  With UID one can be sure to delete exactly the right row:
+
+```
+kal select --todo list --template='{UID} {SUMMARY}'
+```
+
+### Editing and deleting objects
+
+```
+kal select --todo --uid=1234-5678-9abc delete
+kal select --todo --category computer-work --start=2022-04-04 --end=2022-05-05 edit --complete ## not supported yet
+kal select --todo --category computer-work --overdue edit --postpone=5d ## not supported yet
diff --git a/cal.py b/cal.py
index a4f4b19..88ce91b 100755
--- a/cal.py
+++ b/cal.py
@@ -25,6 +25,12 @@ from calendar_cli import __version__
 import click
 import os
 import caldav
+#import isodate
+import dateutil
+import datetime
+import re
+from icalendar import prop
+from lib.template import Template
 
 ## should make some subclasses of click.ParamType:
 
@@ -38,19 +44,89 @@ import caldav
 ## See https://click.palletsprojects.com/en/8.0.x/api/#click.ParamType and
 ## /usr/lib/*/site-packages/click/types.py on how to do this.
 
-## TODO: maybe find those attributes through the icalendar library?
+## TODO: maybe find those attributes through the icalendar library? icalendar.cal.singletons, icalendar.cal.multiple, etc
 attr_txt_one = ['location', 'description', 'geo', 'organizer', 'summary']
-attr_txt_many = ['categories', 'comment', 'contact', 'resources']
+attr_txt_many = ['category', 'comment', 'contact', 'resources']
+
+def parse_dt(input, return_type=None):
+    """Parse a datetime or a date.
+
+    If return_type is date, return a date - if return_type is
+    datetime, return a datetime.  If no return_type is given, try to
+    guess if we should return a date or a datetime.
 
-## TODO ... (and should be moved somewhere else)
-def _parse_timespec(timespec):
     """
-    will parse a timespec and return two timestamps.  timespec can be
-    in ISO8601 interval format, as format 1, 2 or 3 as described at
-    https://en.wikipedia.org/wiki/ISO_8601#Time_intervals - or it can
-    be on calendar-cli format (i.e. 2021-01-08 15:00:00+1h)
+    ret = dateutil.parser.parse(input)
+    if return_type is datetime.datetime:
+        return ret
+    elif return_type is datetime.date:
+        return ret.date()
+    elif ret.time() == datetime.time(0,0) and len(input)<12 and not '00:00' in input and not '0000' in input:
+        return ret.date()
+    else:
+        return ret
+
+def parse_add_dur(dt, dur):
     """
-    raise NotImplementedError()
+    duration may be something on the format 1s (one second), 3m (three minutes, not months), 3.5h, 1y1w, etc
+    or a ISO8601 duration (TODO: not supported yet).  Return the dt plus duration
+    """
+    time_units = {
+        's': 1, 'm': 60, 'h': 3600,
+        'd': 86400, 'w': 604800
+    }
+    while dur:
+        rx = re.match(r'(\d+(?:\.\d+)?)([smhdw])(.*)', dur)
+        assert rx
+        i = float(rx.group(1))
+        u = rx.group(2)
+        dur = rx.group(3)
+        if u=='y':
+            dt = datetime.datetime.combine(datetime.date(dt.year+i, dt.month, dt.day), dt.time())
+        else:
+            dt = dt + datetime.timedelta(0, i*time_units[u])
+    return dt
+   
+
+## TODO ... (and should be moved somewhere else?)
+def parse_timespec(timespec):
+    """parses a timespec and return two timestamps
+
+    The ISO8601 interval format, format 1, 2 or 3 as described at
+    https://en.wikipedia.org/wiki/ISO_8601#Time_intervals should be
+    accepted, though it may be dependent on
+    https://github.com/gweis/isodate/issues/77 or perhaps
+    https://github.com/dateutil/dateutil/issues/1184 
+    
+    The calendar-cli format (i.e. 2021-01-08 15:00:00+1h) should be accepted
+
+    Two timestamps should be accepted.
+
+    One timestamp should be accepted, and the second return value will be None.
+    """
+    ## calendar-cli format, 1998-10-03 15:00+2h
+    if '+' in timespec:
+        rx = re.match(r'(.*)\+((?:\d+(?:\.\d+)?[smhdwy])+)$', timespec)
+        if rx:
+            start = parse_dt(rx.group(1))
+            end = parse_add_dur(start, rx.group(2))
+            return (start, end)
+    try:
+        ## parse("2015-05-05 2015-05-05") does not throw the ParserError
+        if timespec.count('-')>3:
+            raise dateutil.parser.ParserError("Seems to be two dates here")
+        ret = parse_dt(timespec)
+        return (ret,None)
+    except dateutil.parser.ParserError:
+        split_by_space = timespec.split(' ')
+        if len(split_by_space) == 2:
+            return (parse_dt(split_by_space[0]), parse_dt(split_by_space[1]))
+        elif len(split_by_space) == 4:
+            return (parse_dt(f"{split_by_space[0]} {split_by_space[1]}"), parse_dt(f"{split_by_space[2]} {split_by_space[3]}"))
+        else:
+            raise ValueError(f"couldn't parse time interval {timespec}")
+
+    raise NotImplementedError("possibly a ISO time interval")
 
 @click.group()
 ## TODO
@@ -100,22 +176,37 @@ def test(ctx):
     """
     click.echo("Seems like everything is OK")
 
-def _set_attr_options(func):
+def _set_attr_options_(func, verb):
     """
-    decorator that will add options --set-categories, --set-description etc
+    decorator that will add options --set-category, --set-description etc
     """
+    if verb:
+        verb1 = f"{verb}-"
+    else:
+        verb1 = ""
+        verb = "Select by "
     for foo in attr_txt_one:
-        func = click.option(f"--set-{foo}", help=f"Set ical attribute {foo}")(func)
+        func = click.option(f"--{verb1}{foo}", help=f"{verb} ical attribute {foo}")(func)
     for foo in attr_txt_many:
-        func = click.option(f"--set-{foo}", help=f"Set ical attribute {foo}", multiple=True)(func)
+        func = click.option(f"--{verb1}{foo}", help=f"{verb} ical attribute {foo}", multiple=True)(func)
     return func
 
+def _set_attr_options(verb=""):
+    return lambda func: _set_attr_options_(func,verb)
+
 @cli.group()
 @click.option('--all/--none', default=None, help='Select all (or none) of the objects.  Overrides all other selection options.')
-@click.option('--uid', multiple=True, help='select an object with a given uid (or select more object with given uids)')
-@click.option('--abort-on-missing-uid/--ignore-missing-uid', default=False, help='Abort if (one or more) uids are not found (default: silently ignore missing uids)')
+@click.option('--uid', multiple=True, help='select an object with a given uid (or select more object with given uids).  Overrides all other selection options')
+@click.option('--abort-on-missing-uid/--ignore-missing-uid', default=False, help='Abort if (one or more) uids are not found (default: silently ignore missing uids).  Only effective when used with --uid')
+@click.option('--todo/--notodo', default=None, help='select only todos (or no todos)')
+@click.option('--event/--noevent', default=None, help='select only todos (or no todos)')
+@click.option('--include-completed/--exclude-completed', default=False, help='select only todos (or no todos)')
+@_set_attr_options()
+@click.option('--start', help='do a time search, with this start timestamp')
+@click.option('--end', help='do a time search, with this end timestamp (or duration)')
+@click.option('--timespan', help='do a time search for this interval')
 @click.pass_context
-def select(ctx, all, uid, abort_on_missing_uid, **kwargs):
+def select(ctx, all, uid, abort_on_missing_uid, **kwargs_):
     """
     select/search/filter tasks/events, for listing/editing/deleting, etc
     """
@@ -129,15 +220,26 @@ def select(ctx, all, uid, abort_on_missing_uid, **kwargs):
         return
     if all:
         for c in ctx.obj['calendars']:
-            objs.append(c.objects)
+            objs.extend(c.objects())
+        return
+
+    kwargs = {}
+    for kw in kwargs_:
+        if kwargs_[kw] is not None and kwargs_[kw] != ():
+            kwargs[kw] = kwargs_[kw]
 
     ## uid(s)
     missing_uids = []
     for uid_ in uid:
+        comp_filter=None
+        if kwargs['event']:
+            comp_filter='VEVENT'
+        if kwargs_['todo']:
+            comp_filter='VTODO'
         cnt = 0
         for c in ctx.obj['calendars']:
             try:
-                objs.append(c.object_by_uid(uid_))
+                objs.append(c.object_by_uid(uid_, comp_filter=comp_filter))
                 cnt += 1
             except caldav.error.NotFoundError:
                 pass
@@ -145,14 +247,49 @@ def select(ctx, all, uid, abort_on_missing_uid, **kwargs):
             missing_uids.append(uid_)
     if abort_on_missing_uid and missing_uids:
         raise click.Abort(f"Did not find the following uids in any calendars: {missing_uids}")
+    if uid:
+        return
+
+    if kwargs_['start']:
+        kwargs['start'] = parse_dt(kwargs['start'])
+        if kwargs_['end']:
+            rx = re.match(r'\+((\d+(\.\d+)?[smhdwy])+)', kwargs['end'])
+            if rx:
+                kwargs['end'] = parse_add_dur(kwargs['start'], rx.group(1))
+            else:
+                kwargs['end'] = parse_dt(kwargs['end'])
+    elif kwargs_['timespan']:
+        kwargs['start'], kwargs['end'] = parse_timespec(kwargs['timespan'])
+
+    for attr in attr_txt_many:
+        if len(kwargs_[attr])>1:
+            raise NotImplementedError(f"is it really needed to search for more than one {attr}?")
+        elif kwargs_[attr]:
+            kwargs[attr] = kwargs[attr][0]
+
+    for c in ctx.obj['calendars']:
+        objs.extend(c.search(**kwargs))
 
 @select.command()
+@click.option('--ics/--no-ics', default=False, help="Output in ics format")
+@click.option('--template', default="{DUE.dt:?{DTSTART.dt:?(date missing)?}?:%F %H:%M:%S}: {SUMMARY:?{DESCRIPTION:?(no summary given)?}?}")
 @click.pass_context
-def list(ctx, **kwargs):
+def list(ctx, ics, template):
     """
     print out a list of tasks/events/journals
     """
-    raise NotImplementedError()
+    if ics:
+        if not ctx.obj['objs']:
+            return
+        icalendar = ctx.obj['objs'].pop(0).icalendar_instance
+        for obj in ctx.obj['objs']:
+            icalendar.subcomponents.extend(obj.icalendar_instance.subcomponents)
+        click.echo(icalendar.to_ical())
+        return
+    template=Template(template)
+    for obj in ctx.obj['objs']:
+        for sub in obj.icalendar_instance.subcomponents:
+            click.echo(template.format(**sub))
 
 @select.command()
 @click.option('--multi-delete/--no-multi-delete', default=None, help="Delete multiple things without confirmation prompt")
@@ -170,6 +307,28 @@ def delete(ctx, multi_delete, **kwargs):
         obj.delete()
 
 @select.command()
+@click.option('--add-category', default=None, help="Delete multiple things without confirmation prompt", multiple=True)
+@_set_attr_options(verb='set')
+@click.pass_context
+def edit(ctx, add_category=None, **kwargs):
+    _process_set_args(ctx, kwargs)
+    for obj in ctx.obj['objs']:
+        ie = obj.icalendar_instance.subcomponents[0]
+        for arg in ctx.obj['set_args']:
+            if arg in ie:
+                ie.pop(arg)
+            ie.add(arg, ctx.obj['set_args'][arg])
+        if add_category:
+            if 'categories' in ie:
+                cats = ie.pop('categories').cats
+            else:
+                cats = []
+            cats.extend(add_category)
+            ie.add('categories', cats)
+    obj.save()
+
+
+@select.command()
 @click.pass_context
 def complete(ctx, **kwargs):
     raise NotImplementedError()
@@ -189,7 +348,6 @@ def sum_hours(ctx, **kwargs):
 @click.option('-l', '--add-ical-line', multiple=True, help="extra ical data to be injected")
 @click.option('--multi-add/--no-multi-add', default=None, help="Add things to multiple calendars")
 @click.option('--first-calendar/--no-first-calendar', default=None, help="Add things only to the first calendar found")
-@_set_attr_options
 @click.pass_context
 def add(ctx, **kwargs):
     """
@@ -212,10 +370,6 @@ def add(ctx, **kwargs):
             raise click.Abort("Giving up: Multiple calendars found/given, please specify which calendar you want to use")
 
     ctx.obj['ical_fragment'] = "\n".join(kwargs['add_ical_line'])
-    ctx.obj['add_args'] = {}
-    for x in kwargs:
-        if x.startswith('set_'):
-            ctx.obj['add_args'][x[4:]] = kwargs[x]
 
 @add.command()
 @click.pass_context
@@ -230,10 +384,23 @@ def ical(ctx, ical_data, ical_file):
         ## TODO: this may not be an event - should make a Calendar.save_object method
         c.save_event(ical)
 
+def _process_set_args(ctx, kwargs):
+    ctx.obj['set_args'] = {}
+    for x in kwargs:
+        if x == 'set_category' and kwargs[x] != ():
+            ctx.obj['set_args']['categories'] = kwargs[x]
+        elif x.startswith('set_') and kwargs[x] is not None and kwargs[x] != ():
+            ctx.obj['set_args'][x[4:]] = kwargs[x]
+    if 'summary' in kwargs:
+        ctx.obj['set_args']['summary'] = ctx.obj['set_args'].get('summary', '') + kwargs['summary']
+    if kwargs['ical_fragment']:
+        ctx.obj['set_args']['ics'] = kwargs['ical_fragment']
+
 @add.command()
 @click.argument('summary', nargs=-1)
+@_set_attr_options(verb='set')
 @click.pass_context
-def todo(ctx, summary):
+def todo(ctx, **kwargs):
     """
     Creates a new task with given SUMMARY
 
@@ -242,21 +409,22 @@ def todo(ctx, summary):
     kal add todo "fix all known bugs in calendar-cli"
     kal add todo --set-due=2050-12-10 "release calendar-cli version 42.0.0"
     """
-    summary = " ".join(summary)
-    ctx.obj['add_args']['summary'] = (ctx.obj['add_args']['summary'] or '') + summary
-    if not ctx.obj['add_args']['summary']:
+    kwargs['summary'] = " ".join(kwargs['summary'])
+    _process_set_args(ctx, kwargs)
+    if not ctx.obj['set_args']['summary']:
         raise click.Abort("denying to add a TODO with no summary given")
         return
     for cal in ctx.obj['calendars']:
-        todo = cal.save_todo(ical=ctx.obj['ical_fragment'], **ctx.obj['add_args'], no_overwrite=True)
+        todo = cal.save_todo(ical=ctx.obj['ical_fragment'], **ctx.obj['set_args'], no_overwrite=True)
         click.echo(f"uid={todo.id}")
 
 @add.command()
 ## TODO
 @click.argument('summary')
 @click.argument('timespec')
+@_set_attr_options(verb='set')
 @click.pass_context
-def event(ctx, summary, timespec):
+def event(ctx, timespec, **kwargs):
     """
     Creates a new event with given SUMMARY at the time specifed through TIMESPEC.
 
@@ -267,10 +435,11 @@ def event(ctx, summary, timespec):
     kal add event "final bughunting session" 2004-11-25+5d
     kal add event "release party" 2004-11-30T19:00+2h
     """
+    _process_set_args(ctx, kwargs)
     for cal in ctx.obj['calendars']:
-        (dtstart, dtend) = _parse_timespec(timespec)
-        #event = cal.add_event(
-        click.echo(f"uid={uid}")
+        (dtstart, dtend) = parse_timespec(timespec)
+        event = cal.save_event(dtstart=dtstart, dtend=dtend, **ctx.obj['set_args'], no_overwrite=True)
+        click.echo(f"uid={event.id}")
 
 def journal():
     click.echo("soon you should be able to add journal entries to your calendar")
diff --git a/lib/template.py b/lib/template.py
new file mode 100644
index 0000000..47281d9
--- /dev/null
+++ b/lib/template.py
@@ -0,0 +1,49 @@
+"""String formatter that allows default values to be passed in the
+template string.
+
+This does not really belong in the calendar-cli package.  I was
+googling a bit, and didn't find anything like this out there ... but
+I'm sure there must exist something like this?"""
+
+import datetime
+import string
+import re
+
+class NoValue():
+    def __getattr__(self, attr):
+        return self
+    def __getitem__(self, attr):
+        return self
+    def __str__(self):
+        return ""
+    def __format__(self, spec):
+        try:
+            return "".__format__(spec)
+        except:
+            return ""
+
+no_value = NoValue()
+
+class Template(string.Formatter):
+    def __init__(self, template):
+        self.template = template
+
+    def format(self, *pargs, **kwargs):
+        return super().format(self.template, *pargs, **kwargs)
+    
+    def get_value(self, key, args, kwds):
+        try:
+            return string.Formatter.get_value(self, key, args, kwds)
+        except:
+            return no_value
+
+    def format_field(self, value, format_spec):
+        rx = re.match(r'\?([^\?]*)\?(.*)', format_spec)
+        if rx:
+            format_spec = rx.group(2)
+            if value is no_value:
+                value = rx.group(1)
+        try:
+            return string.Formatter.format_field(self, value, format_spec)
+        except:
+            return string.Formatter.format_field(self, value, "")
diff --git a/setup.py b/setup.py
index dbd6bd6..0d91b62 100644
--- a/setup.py
+++ b/setup.py
@@ -32,7 +32,8 @@ setup(
     py_modules=['cal'],
     install_requires=[
         'icalendar',
-        'caldav>=0.8.1',
+        'caldav>=0.10.0dev',
+#        'isodate',
         'pytz', ## pytz is supposed to be obsoleted, but see https://github.com/collective/icalendar/issues/333 
         'tzlocal',
         'Click',
diff --git a/tests/test_cal.py b/tests/test_cal.py
new file mode 100644
index 0000000..bfbd325
--- /dev/null
+++ b/tests/test_cal.py
@@ -0,0 +1,131 @@
+import pytest
+import sys
+sys.path.insert(0,'.')
+sys.path.insert(1,'..')
+from datetime import datetime, date
+from cal import parse_timespec
+from lib.template import Template
+
+"""calendar-cli is a command line utility, and it's an explicit design
+goal that it should contain minimal logic except for parsing and
+passing command line options and parameters to the caldav library and
+printing output from the caldav library.  Functional tests verifying
+that the tool actually works as intended is done through shell
+scripts, and can be run through test_calendar-cli.sh.  There is no
+goal to get complete code coverage through this unit test, though any
+"extra" logic except for simple passing of options and parameters to
+the caldav library ought to be tested here.  """
+
+class TestTemplate:
+    def setup(self):
+        self.date = date(1990, 10, 10)
+        
+    def test_formatting_with_timespec(self):
+        template=Template("This is an ISO date: {date:%F}")
+        text = template.format(date=self.date)
+        assert text == "This is an ISO date: 1990-10-10"
+
+        text = template.format(foo=self.date)
+        assert text == "This is an ISO date: "
+        
+    def test_formatting_with_simple_default(self):
+        template=Template("This is an ISO date: {date:?(date is missing)?%F}")
+        text = template.format(date=self.date)
+        assert text == "This is an ISO date: 1990-10-10"
+
+        text = template.format(foo=self.date)
+        assert text == "This is an ISO date: (date is missing)"
+
+    def test_subvalue_with_default(self):
+        template = Template("This is a year: {date.year:?NA?>5}")
+        text = template.format(date=self.date)
+        assert text == "This is a year:  1990"
+        text = template.format(foo=self.date)
+        assert text == "This is a year:    NA"
+
+    def test_missing_replaced_with_advanced_default(self):
+        template = Template("Date is maybe {date:?{foo}?%F}")
+        text = template.format(date=self.date)
+        assert text == "Date is maybe 1990-10-10"
+        text = template.format(foo=self.date)
+        assert text == "Date is maybe 1990-10-10"
+        text = template.format(foo=self.date, date=self.date)
+        assert text == "Date is maybe 1990-10-10"
+
+    def test_missing_replaced_with_even_more_advanced_default(self):
+        template = Template("Date is maybe {date:?{foo:?bar?}?%F}")
+        text = template.format(date=self.date)
+        assert text == "Date is maybe 1990-10-10"
+        text = template.format(foo=self.date)
+        assert text == "Date is maybe 1990-10-10"
+        text = template.format(foo=self.date, date=self.date)
+        assert text == "Date is maybe 1990-10-10"
+        text = template.format()
+        assert text == "Date is maybe bar"
+
+class TestParseTimestamp:
+    def _testTimeSpec(self, expected):
+        for input in expected:
+            assert parse_timespec(input) == expected[input]
+
+    @pytest.mark.skip(reason="Not implemented yet, waiting for feedback on https://github.com/gweis/isodate/issues/77")
+    def testIsoIntervals(self):
+        raise pytest.SkipTest("")
+        expected = {
+            "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z":
+                (datetime(2007,3,1,13), datetime(2008,5,11,15,30)),
+            "2007-03-01T13:00:00Z/P1Y2M10DT2H30M":
+                (datetime(2007,3,1,13), datetime(2008,5,11,15,30)),
+            "P1Y2M10DT2H30M/2008-05-11T15:30:00Z":
+                (datetime(2007,3,1,13), datetime(2008,5,11,15,30))
+        }
+        self._testTimeSpec(expected)
+
+    def testOneTimestamp(self):
+        expected = {
+            "2007-03-01T13:00:00":
+                (datetime(2007,3,1,13), None),
+            "2007-03-01 13:00:00":
+                (datetime(2007,3,1,13), None),
+        }
+        self._testTimeSpec(expected)
+        
+    def testOneDate(self):
+        expected = {
+            "2007-03-01":
+                (date(2007,3,1), None)
+        }
+        self._testTimeSpec(expected)
+        
+    def testTwoTimestamps(self):
+        expected = {
+            "2007-03-01T13:00:00 2007-03-11T13:30:00":
+                (datetime(2007,3,1,13), datetime(2007,3,11,13,30)),
+            "2007-03-01 13:00:00 2007-03-11 13:30:00":
+                (datetime(2007,3,1,13), datetime(2007,3,11,13,30)),
+        }
+        self._testTimeSpec(expected)
+
+    def testTwoDates(self):
+        expected = {
+            "2007-03-01 2007-03-11":
+                (date(2007,3,1), date(2007,3,11))
+        }
+        self._testTimeSpec(expected)
+
+    def testCalendarCliFormat(self):
+        expected = {
+            "2007-03-01T13:00:00+10d":
+                (datetime(2007,3,1,13), datetime(2007,3,11,13)),
+            "2007-03-01T13:00:00+2h":
+                (datetime(2007,3,1,13), datetime(2007,3,1,15)),
+            "2007-03-01T13:00:00+2.5h":
+                (datetime(2007,3,1,13), datetime(2007,3,1,15,30)),
+            "2007-03-01T13:00:00+2h30m":
+                (datetime(2007,3,1,13), datetime(2007,3,1,15,30)),
+            "2007-03-01+10d":
+                (date(2007,3,1), date(2007,3,11))
+        }
+        self._testTimeSpec(expected)
+
+        
diff --git a/tests/test_calendar-cli.sh b/tests/test_calendar-cli.sh
index b45d596..8e408e2 100755
--- a/tests/test_calendar-cli.sh
+++ b/tests/test_calendar-cli.sh
@@ -3,7 +3,7 @@
 storage=$(mktemp -d)
 
 echo "This script will attempt to set up a Radicale server and a Xandikos server and run the test code towards those two servers"
-echo "The test code itself is found in tests.sh"
+echo "The test code itself is found in tests.sh and tests_kal.sh"
 
 export RUNTESTSNOPAUSE="foo"
 
@@ -31,7 +31,8 @@ then
 	echo "press enter to run tests"
 	read foo
     fi
-    ./tests.sh
+    ./tests_kal.sh
+    #./tests.sh
     if [ -n "$DEBUG" ]
     then
 	echo "press enter to take down test server"
@@ -62,7 +63,8 @@ then
     kal="../cal.py  --caldav-url=http://localhost:8080/ --caldav-user=user"
     export calendar_cli
     export kal
-    ./tests.sh
+    ./tests_kal.sh
+    #./tests.sh
     kill $xandikos_pid
 else
     echo "## Could not start up xandikos (is it installed?).  Will skip running tests towards xandikos"
diff --git a/tests/tests.sh b/tests/tests.sh
index bd1fbfb..1c91270 100755
--- a/tests/tests.sh
+++ b/tests/tests.sh
@@ -22,7 +22,6 @@ then
     echo "This script will use the following commands to access a calendar server:"
     echo
     echo "$calendar_cli"
-    echo "$kal"
     echo
     echo "This may work if you have configured a calendar server."
     echo "The tests will add and delete events and tasks."
@@ -77,7 +76,7 @@ echo "## Deleting events with uid $uid $uid1 $uid2"
 calendar_cli calendar delete --event-uid=$uid
 calendar_cli calendar delete --event-uid=$uid2
 calendar_cli calendar delete --event-uid=$uid3
-kal select --uid=$uid3 delete
+
 echo "## Searching again for the deleted event"
 calendar_cli calendar agenda --from-time=2010-10-10 --agenda-days=3
 echo $output | { grep -q 'testing testing' && error "still found the event" ; } || echo "## OK: didn't find the event"
@@ -103,7 +102,7 @@ echo "## cleanup, delete it"
 calendar_cli calendar delete --event-uid=$uid
 
 echo "## Same, using kal add ics"
-kal add ical --ical-file=$tmpfile
+calendar_cli calendar  addics --file $tmpfile
 
 rm $tmpfile
 calendar_cli  --icalendar calendar agenda --from-time=2010-10-13 --agenda-days=1
@@ -115,8 +114,6 @@ echo "## cleanup, delete it"
 calendar_cli calendar delete --event-uid=$uid
 
 ## TODO - procrastinated, waiting for response on https://github.com/dateutil/dateutil/issues/1184 
-#echo "## Same, using kal add event"
-#kal add event '2010-10-10+4d' 'whole day testing'
 #uid=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
 #[ -n "$uid" ] || error "got no UID back"
 #calendar_cli  --icalendar calendar agenda --from-time=2010-10-13 --agenda-days=1
@@ -184,7 +181,7 @@ echo "## TODOS / TASK LISTS"
 echo "## Attempting to add a task with category 'scripttest'"
 calendar_cli todo add --set-categories scripttest "edit this task"
 uidtodo1=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
-kal add --set-categories scripttest todo "edit this task2"
+calendar_cli todo add --set-categories scripttest todo "edit this task2"
 uidtodo2=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
 
 echo "## Listing out all tasks with category set to 'scripttest'"
@@ -193,7 +190,6 @@ calendar_cli todo --categories scripttest list
 
 echo "## Editing the task"
 calendar_cli todo --categories scripttest edit --set-summary "editing" --add-categories "scripttest2"
-#kal select --todo --categories scripttest edit --add-categories "scripttest3"
 
 echo "## Verifying that the edits got through"
 calendar_cli todo --categories scripttest list
diff --git a/tests/tests_kal.sh b/tests/tests_kal.sh
new file mode 100755
index 0000000..3858063
--- /dev/null
+++ b/tests/tests_kal.sh
@@ -0,0 +1,218 @@
+#!/bin/bash
+
+## TODO: all references to calendar-cli should be replaced with references to kal.  Work in progress!
+
+set -e
+
+########################################################################
+## SETUP
+########################################################################
+
+for path in . .. ./tests ../tests
+do
+    setup="$path/_setup_alias"
+    [ -f $setup  ] && source $setup
+done
+
+if [ -z "$RUNTESTSNOPAUSE" ]
+then
+    echo "tests.sh"
+    echo
+    echo "Generally, tests.sh should only be run directly if you know what you are doing"
+    echo "You may want to use test_calendar-cli.sh instead"
+    echo
+    echo "This script will use the following commands to access a calendar server:"
+    echo
+    echo "$kal"
+    echo "$calendar_cli"
+    echo
+    echo "This may work if you have configured a calendar server."
+    echo "The tests will add and delete events and tasks."
+    echo "Content from 2010-10 may be deleted"
+    echo
+    echo "Press enter or ctrl-C"
+    read foo
+fi
+
+echo "## CLEANUP from earlier failed test runs, if any"
+
+QUIET=true
+for uid in $($calendar_cli calendar agenda --from-time=2010-10-09 --agenda-days=5 --event-template='{uid}') ; do calendar_cli calendar delete --event-uid=$uid ; done
+calendar_cli todo --categories scripttest delete
+unset QUIET
+
+########################################################################
+## TEST CODE FOLLOWS
+########################################################################
+
+echo "## EVENTS"
+
+echo "## this is a very simple test script without advanced error handling"
+echo "## if this test script doesn't output 'ALL TESTS COMPLETED!  YAY!' in the end, something went wrong"
+
+echo "## Attempting to add an event at 2010-10-09 20:00:00, 2 hours duration"
+kal add event 'testing testing' '2010-10-09 20:00:00+2h' 
+uid=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+[ -n "$uid" ] || error "got no UID back"
+
+echo "## Attempting to add an event at 2010-10-10 20:00:00, CET (1 hour duration is default), with description and non-ascii location"
+kal add event 'testing testing' '2010-10-10 20:00:00+01:00' --set-description='this is a test calendar event' --set-location='Москва'
+uid2=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+[ -n "$uid2" ] || error "got no UID back"
+
+echo "## Attempting to add an event at 2010-10-11 20:00:00, CET, 3h duration"
+kal add event 'testing testing' '2010-10-11 20:00:00+01:00+3h'
+uid3=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+echo "## OK: Added the event, uid is $uid"
+
+echo "## Taking out the agenda for 2010-10-09 + four days"
+kal select --start=2010-10-09 --end=+4d --event list --template='{DESCRIPTION} {LOCATION}'
+echo $output | { grep -q 'this is a test calendar event Москва' && echo "## OK: found the event" ; } || error "didn't find the event"
+
+echo "## Taking out the agenda for 2010-10-10, with uid"
+kal select --start=2010-10-10 --end=+1d --event list --template='{DTSTART.dt} {UID}'
+echo $output | { grep -q $uid2 && echo "## OK: found the UID" ; } || error "didn't find the UID"
+
+echo "## Deleting events with uid $uid $uid2 $uid3"
+kal select --event --uid=$uid delete
+kal select --event --uid=$uid2 delete
+kal select --event --uid=$uid3 delete
+
+echo "## Searching again for the deleted event"
+kal select --event --start=2010-10-10 --end=+3d list
+echo $output | { grep -q 'testing testing' && error "still found the event" ; } || echo "## OK: didn't find the event"
+
+echo "## Adding a full day event"
+kal add event 'whole day testing' '2010-10-10+4d' 
+uid=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+[ -n "$uid" ] || error "got no UID back"
+
+echo "## fetching the full day event, in ics format"
+kal select --start=2010-10-13 --end=+1d --event list --ics
+
+echo "$output" | grep -q "whole day" || error "could not find the event"
+echo "$output" | grep -q "20101010" || error "could not find the date"
+echo "$output" | grep -q "20101010T" && error "a supposed whole day event was found to be with the time of day"
+echo "OK: found the event"
+
+## saving the ics data
+tmpfile=$(mktemp)
+cat $outfile > $tmpfile
+
+echo "## cleanup, delete it"
+kal select --event --uid=$uid delete
+
+echo "## Same, using kal add ics"
+kal add ical --ical-file=$tmpfile
+rm $tmpfile
+
+kal select --event --start=2010-10-13 --end=2010-10-14 list --ics
+echo "$output" | grep -q "whole day" || error "could not find the event"
+echo "$output" | grep -q "20101010" || error "could not find the date"
+echo "$output" | grep -q "20101010T" && error "a supposed whole day event was found to be with the time of day"
+echo "$output" | grep UID
+echo "OK: found the event"
+echo "## cleanup, delete it"
+
+kal select --event --uid=$uid delete
+
+## TODO: PROCRASTINATING TIME ZONES.  Waiting for a release of the icalendar library that doesn't depend on pytz
+if [ -n "" ]; then
+echo "## testing timezone support"
+echo "## Create a UTC event"
+calendar_cli --timezone='UTC' calendar add '2010-10-09 12:00:00+10m' 'testevent with a UTC timezone'
+uid=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+[ -n "$uid" ] || error "got no UID back"
+
+echo "## fetching the UTC-event, as ical data"
+calendar_cli --icalendar --timezone=UTC calendar agenda --from-time='2010-10-09 11:59' --agenda-mins=3
+[ -n "$output" ] || error "failed to find the event that was just added"
+echo "$output" | grep -q "20101009T120000Z" || error "failed to find the UTC timestamp.  Perhaps the server is yielding timezone data for the UTC timezone?  In that case, the assert in the test code should be adjusted"
+
+echo "## cleanup, delete it"
+calendar_cli calendar delete --event-uid=$uid
+
+echo "## Create an event with a somewhat remote time zone, west of UTC"
+calendar_cli --timezone='Brazil/DeNoronha' calendar add '2010-10-09 12:00:00+10m' 'testevent with a time zone west of UTC'
+uid=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+[ -n "$uid" ] || error "got no UID back"
+
+echo "## fetching the remote time zone event, as ical data"
+calendar_cli --icalendar --timezone=UTC calendar agenda --from-time='2010-10-09 13:59' --agenda-mins=3
+## zimbra changes Brazil/DeNoronha to America/Noronha.  Actually, the server may theoretically use arbitrary IDs for the timezones.
+echo "$output" | grep -Eq "TZID=\"?[a-zA-Z/]*Noronha" || echo "$output" | grep -q "140000Z" ||
+    error "failed to find the remote timezone"
+
+echo "## fetching the remote time zone event, in UTC-time"
+calendar_cli --timezone=UTC calendar agenda --from-time='2010-10-09 13:59' --agenda-mins=3 --event-template='{dtstart}'
+[ "$output" == '2010-10-09 14:00 (Sat)' ] || error "expected dtstart to be 2010-10-09 14:00 (Sat)"
+
+echo "## fetching the remote time zone event, in CET-time (UTC+2 with DST, and October is defined as summer in Oslo, weird)"
+calendar_cli --timezone=Europe/Oslo calendar agenda --from-time='2010-10-09 15:59' --agenda-mins=3 --event-template='{dtstart}'
+[ "$output" == '2010-10-09 16:00 (Sat)' ] || error "expected dtstart to be 2010-10-09 15:00 (Sat)"
+
+echo "## cleanup, delete it"
+calendar_cli calendar delete --event-uid=$uid
+fi
+
+echo "## TODOS / TASK LISTS"
+
+echo "## Attempting to add a task with category 'scripttest'"
+kal add todo --set-category scripttest "edit this task"
+uidtodo1=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+kal add todo --set-category scripttest "edit this task2"
+uidtodo2=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+
+echo "## Listing out all tasks with category set to 'scripttest'"
+kal select --todo --category scripttest list
+[ $(echo "$output" | wc -l) == 2 ] || error "We found more or less or none of the two todo items we just added"
+
+echo "## Editing the task"
+kal select --todo --category scripttest edit --set-summary "editing" --add-category "scripttest2"
+
+echo "## Verifying that the edits got through"
+kal select --todo --category scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found the todo item we just edited and nothing more"
+kal select --todo --category scripttest2 list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found the todo item we just edited and nothing more"
+kal select --todo --category scripttest3 list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found the todo item we just edited and nothing more"
+kal select --todo --comment editing list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found the todo item we just edited and nothing more"
+
+if [ -n "" ]; then
+echo "## Complete the task"
+calendar_cli todo --categories scripttest complete
+calendar_cli todo --categories scripttest list
+[ -z "$output" ] && echo "## OK: todo-item is done"
+calendar_cli todo --todo-uid $uidtodo1 delete
+
+## parent-child relationships
+echo "## Going to add three todo-items with children/parent relationships"
+calendar_cli todo add --set-categories scripttest "this is a grandparent"
+uidtodo2=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories=scripttest add --set-categories scripttest --is-child "this is a parent and a child"
+uidtodo3=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories=scripttest add --set-categories scripttest --is-child "this task has two parents"
+uidtodo4=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories scripttest list
+[ $(echo "$output" | wc -l) == 3 ] && echo "## OK: found three tasks"
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+echo "## Going to complete the children task"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ -z "$output" ] && echo "## OK: found no tasks now"
+
+fi
+
+echo "## some kal TESTS COMPLETED SUCCESSFULLY!  YAY!"
+
+rm $outfile
+