Merge branch 'v0.11.0.dev' with conflict handling

5 files changed, 322 insertions, 68 deletions

diff --git a/EXAMPLES b/EXAMPLES
new file mode 100644
index 0000000..4d5b375
--- /dev/null
+++ b/EXAMPLES
@@ -0,0 +1,25 @@
+TODO: clean this up a bit
+
+Eventually I'd like to make calendar-cli easier to use, so some of the "external logic" in those examples will eventually be moved into the tool itself.  Anyway, some of the power of having a command-line utility is that it's possible to do just anything ...
+
+## Take out a personal agenda from different calendar sources:
+for section in pp seb house default work holidays ; do ./calendar-cli.py --config-section $section calendar agenda  --agenda-days 20; done | sort ; for section in seb house default work-tasks pp-tasks ; do echo $section; ./calendar-cli.py --config-section $section todo --limit 10 --hide-parent list ; done ## AGENDA
+
+## Interactively set categories on uncategorized tasks:
+cd ~/calendar-cli/ ; { ./calendar-cli.py todo list --list-categories | perl -pe 's/^/# /' ; ./calendar-cli.py todo --nocategories list --todo-template='./calendar-cli.py todo --todo-uid={uid} edit --set-categories=foo # {summary}' ; }  > /tmp/nocat ; $EDITOR /tmp/nocat ; . /tmp/nocat
+
+## Interactively mark tasks as completed:
+cd ~/calendar-cli/ ; { ./calendar-cli.py todo --hide-parents --limit 10 list --todo-template='# ./calendar-cli.py todo --todo-uid={uid} complete # {summary}' ; }  > /tmp/tocomplete ; $EDITOR /tmp/tocomplete ; . /tmp/tocomplete
+
+## Interactively mark tasks as completed, with category:
+CAT=keyboard ; cd ~/calendar-cli/ ; { ./calendar-cli.py todo --hide-parents --categories=$CAT --limit 10 list --todo-template='# ./calendar-cli.py todo --todo-uid={uid} complete # {summary}' ; }  > /tmp/tocomplete ; $EDITOR /tmp/tocomplete ; . /tmp/tocomplete
+
+## Set more realistic due-dates on overdue tasks
+cd ~/calendar-cli/ ; { ./calendar-cli.py todo --overdue list --todo-template='./calendar-cli.py todo --todo-uid={uid} postpone --due "in 2d" # {summary}' ; }  > /tmp/toprocrastinate ; $EDITOR /tmp/toprocrastinate ; . /tmp/toprocrastinate
+
+## Clean the list a bit by procrastinating tasks (this includes the overdue)
+cd ~/calendar-cli/ ; { ./calendar-cli.py todo --hide-future list --todo-template='./calendar-cli.py todo --todo-uid={uid} postpone "in 5d" # {summary}' ; }  > /tmp/toprocrastinate ; $EDITOR /tmp/toprocrastinate ; . /tmp/toprocrastinate
+
+## Simple sync of a google calendar into personal calendar
+wget -O- https://www.google.com/calendar/ical/gsmk.gcal%40gmail.com/public/basic.ics | calendar-cli calendar addics
+
diff --git a/README.md b/README.md
index 056a43d..b875b09 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
 calendar-cli
 ============
 
-Simple command-line CalDav client, for adding and browsing calendar items, todo list items, etc
+Simple command-line CalDav client, for adding and browsing calendar items, todo list items, etc.  As of version 0.11 it's even becoming a full-fledged task management tool.
 
 There is a "competing" project at https://github.com/geier/khal - you may want to check it out - it's more mature but probably more complex.  It's using a "vsyncdir" backend - if I've understood it correctly, that involves building a local copy of the calendar.  The philosophy behind calendar-cli is slightly different, calendar-cli is supposed to be a simple cli-based caldav+ical client.  No synchronization, no local storage.
 
diff --git a/TASK_MANAGEMENT.md b/TASK_MANAGEMENT.md
index 977b2b1..326c73b 100644
--- a/TASK_MANAGEMENT.md
+++ b/TASK_MANAGEMENT.md
@@ -5,10 +5,12 @@ While the RFC does draw some lines on what fields are admissable in the todo-ent
 
 As of 2015-04, this document is just a collection of random thoughts on how to organize task lists.  I haven't done much research in how different software packages handles tasks, nor do I have much experience with managing task lists.  Also, calendar-cli is not really ready yet.
 
-Calendar
---------
+Calendar scope
+--------------
+
+Different categories of tasks can be put into different calendars (and even different caldav servers).
 
-When should you make another calendar/task list and when does it make sense to keep things in the same calendar?
+I believe it's best to keep as few calendars as possible, and rather use i.e. the categories field for splitting different types of tasks.
 
 As you can give access rights to other people for a whole caldav calendar (or "task list"), it makes sense to use the calendar level to control access rights.  You would typically like to have one calendar where your family can view/add tasks, other for work, perhaps separate calendars for separate projects at work if different projects involves different people, etc.
 
@@ -17,7 +19,7 @@ Location
 
 A named location.
 
-Some tasks are only possible to do at a specific location.  When being at that location, one would like to list out the pending tasks that applies for that location.
+One thought - some tasks are only possible to do at a specific location.  When being at that location, one would like to list out the pending tasks that applies for that location.
 
 Examples:
 
@@ -26,58 +28,102 @@ Examples:
 
 Would it make sense to have a separate calendar for the boat?  If you'd like to share the task list with your family, then I think it can be in the same calendar as other family stuff.
 
+Practical experience: This fits much better into the categories field.
+
+Second thought - some todo-items may need to be done at a specific location.  The address can be added to the location field, so it will be visible when inspecting the event at a later stage.  (I've often had a need for this with events, but not with todo items).
+
 Geo
 ---
 
 A geo is a location given by coordinates.  It makes great sense to use geo ...
 
 * if you want to stick the tasks to a map.  Probably very useful if your tasks have to be done on lots of different locations (i.e. if you are a travelling salesman or a plumber).
-* if you want to set up the phone to automatically remind you about tasks i.e. when you are close to the supermarked, etc. 
+* if you want to set up the phone to automatically remind you about tasks i.e. when you are close to the supermarked, etc.
+
+Practical experience: I haven't used the geo field myself.
 
 Categories
 ----------
 
 I'd like to think of categories as tags that can be stuck to tasks.  I.e., some tasks should be done while sitting by the keyboard.  Some tasks are related to a particular project.  Some tasks are best done when the weather is good.  Some tasks has to be done in the day time, others in the evening.  Now, add tags, so that whenever you have the chance to do some task in good weather during the daytime, you can filter out those tasks.
 
-When to use location or geo, and when to use category?  I think that for the super market example, geo is not really fitting because it can only be one geo coordinate related to a vtodo, but there are many super markeds that can be visited.  One could also think that "supermarked" is not a good location for the same reason.
+When to use location or geo, and when to use category?  I think that for the super market example, geo is not really fitting because it can only be one geo coordinate related to a vtodo, but there are many super markeds that can be visited.  One could also think that "supermarked" is not a good location for the same reason.  In practice, I've never used location and geo, always been sticking such information into the categories instead.
 
 Pending-Dependent
 -----------------
 
-If one task A cannot be done without task B being done first, we say that A depends on B.  It may make sense to hide A from todolists, or maybe fade it away.  It may also make sense to push the due date for B such that there is a chance to get A done before it's due time.
+If task A cannot be done without task B being done first, we say that A depends on B.  It may make sense to hide A from todolists, or maybe fade it away.  It may also make sense to ensure the due date for B is before the due date for A.
 
-The VTODO-standard does not support this kind of relationship, but it's possible to use parent-child.  The parent will then be the dependent, and the child will be the pending.
+The VTODO-standard does not support this kind of relationship, but it's possible to use parent-child.  The parent will then be the dependent, and the child will be the pending.  See below for practical experiences.
 
 Parent-child relationship
 -------------------------
 
-This is not supported by calendar-cli as of today - but one can make a hierarchical task list.  It makes a lot of sense when having a big task that can be split up in subtasks.  Say, the task may be "build a bicycle shed".  That does take quite some planning, purchases and work, so one will definitively want to break it up in subtasks.  Ordering such a thing by categories is probably not so productive.
+With the parent-child relationship one can make a hierarchical task list.  It makes a lot of sense when having a big task that can be split up in subtasks.  Say, the task may be "build a bicycle shed".  That does take quite some planning, purchases and work, so one will definitively want to break it up in subtasks.  Ordering such a thing by categories is probably not so productive.  This is more or less compatible with the "Pending-Dependent"-situation above; the task "build a bicycle shed" is dependent on "buy some planks", one would need to buy planks before building the bicycle shed.
 
-What about the shopping list?  "Buy squash" seems to be a subtask of "buy vegetables" which again may be a subtask of "go shopping at the supermarket" - but I think it makes more sense to use categories for that purpose.  There are two differences between the supermarket shopping and the bicycle shed ...
+What about the shopping list?  "Buy squash" seems to be a subtask of "buy vegetables" which again may be a subtask of "go shopping at the supermarket".  From a pending-dependent-perspective it still sort of checks out; you could say that one need to "go shopping" before one can "buy squash", but atoh one cannot successfully complete the "go shopping" without buying the squash.
 
-* Building the bicycle shed serves a purpose for it's own sake.  You're going to buy planks for building the shed, you're not building a shed to buy planks.
+The causality is turned on it's head in the shopping example - the purpose of "go shopping" is to "buy squash", the purpose of "build bike shed" is not to "buy planks".
 
-* "Go to the shop" is not a task for it's own sake - you'd probably not first consider "I need to go to the shop, let's add that to the task list" and then later "According to my task list, I need to go to the shop.  Let's try to make a list of what I need there".  It's more likely that you discover you're running out of sugar and then decide to add "buy sugar" to the shopping list - and you'll go to the shop because you need to buy sugar, you're not buying sugar because you need to go to the shop.
+I'd first add "build a shed" on the todo-list and then try to plan and see what subtasks are needed - but wrg of the sugar, I'd start with "bake a cake", then "buy sugar" and only then I would add "go shopping" to the todo-list.  (Though, my wife would probably first add "go to the shop" and then start thinking what to buy in the shop).
 
-* Multiple parents may not be that trivial - you may have many different projects that requires you to buy planks.
+Practical experience:
+
+* I've been using "supermarket" and "hardware shopping" as categories.  This have been working out fine for me, it makes much more sense than to have "supermarket shopping" and "hardware shopping" as tasks on the list.
+* I never felt a compelling need to group the shopping lists inside the calendar.  On big shopping trips it makes sense to do that, but I'd typically do it externally (i.e. one of the shops I frequently go to - Biltema - I'll make the shopping list inside their web interface, then I get it out with shelf location, information if they are out of stock, prices, etc).
+* Although I haven't created any bike-sheds, I've had some "projects".  First I toss the project into the task-list, with the categories "keyboard" and "thinking".  Later I take up that task and I start creating sub-tasks.  The project then disappears from my regular overview because it has unresolved dependencies.  This has worked out reasonably well for me.
+
+It must be said that parent-child-relationships aren't very well supported yet in calendar-cli.
 
 Recurring tasks
 ---------------
 
 The standard allows for recurring tasks, but doesn't really flesh out what it means that a task is recurring - except that it should show up on date searches if any of the recurrances are within the date search range.
 
-There are two kind of recurrances - should it be done some fixed period after previous time it was done, or should it be done at some fixed period, regardless of when it was done previous time?  Say, the floor should be cleaned weekly.  You usually do it every Monday, but one week everything is so hectic that you postpone it all until Sunday.  When should the floor be washed next time?  If you actually get paid for washing the floor and you have a contract stating that you get paid a weekly sum for washing the floor weekly, then you'd probably want to wash the floor again on Sunday, even if it has been done quite recently.  Except for that, it probably doesn't make sense at all to wash again on Sunday.  You'd probably wait a whole week and wash again next Sunday.
+There are two kind of recurrances:
+
+* Specified intervals - say, the floor should be cleaned every week.  You usually do it every Monday, but one week everything is so hectic that you postpone it all until late Sunday evening.  It would be irrational to wash it again the next day.
+* Fixed-time.  If you actually get paid for washing the floor and you have a contract stating that you get paid a weekly sum for washing the floor weekly, then you'd probably want to wash the floor again on Monday, even if it has been done just recently.
+
+There can be only one status and one complete-date for a vtodo, no matter if it's recurring or not.
 
-There can be only one status and one complete-date for a vtodo, no matter if it's recurring or not.  My idea is to let the client code (calendar-cli - or maybe even the caldav library) duplicate the vtodo, the new vtodo gets the next applicable date, the old vtodo is resolved as completed.
+Based on my interpretation of the standards, I've implemented logic in calendar-cli task completion code to duplicate the vtodo entry if it has an rrule; one vtodo ends up as completed, the other gets a new timestamp based on the rrule ("next after today".  An rrule may be set up both with fixed-time (as in "every Monday, at 10:00") and with specified intervals ("weekly"), so if you complete the task sunday evening, it will be due again Monday if it's a fixed-time rule and Sunday evening if it's with specified intervals).  The two rrules are linked togheter through the recurrance-id attribute.
 
-dtstart vs due vs duration
---------------------------
+There is no support for rrules outside the task completion code, so as for now the rrule has to be put in through another caldav client tool, through the --pdb option or through manually editing ical code.  I believe recurring tasks is an important functionality, so I will implement better support for this at some point.
 
-I my opinion, dtstart is the earliest time you expect to start working with the vtodo, maybe even the earliest time it's possible to start.  One may want to postpone dtstart frequently.
+dtstart vs due vs duration vs priority
+--------------------------------------
 
-due is the time/date when the task has to be completed, come hell or high water.  It should (in most cases) not be postponed.
+I my opinion, dtstart is the earliest time you expect to start working with the vtodo, or maybe the earliest time it's possible to start.  Passing the dtstart doesn't mean you need to drop everything else and start working on the task immediately.  You'd want to postpone the dtstart just to unclutter the todo-list.
+
+Due is the time/date when the task has to be completed, come hell or high water.  It should probably not be postponed.  Due dates should probably be set very far in the future if no specific due date is set.  You really don't want the list of tasks that are soon due or even overdue to be cluttered up with stuff that can be procrastinated even further.
 
 Different task list implementations may behave differently, most of them is probably only concerned with the due date.
 
-According to the RFC, either due or duration should be given, never both of them.  A dtstart and a duration should be considered as equivalent with a dtstart and a due date (where due = dtstart + duration).  I find that a bit sad, I'd like to use dtstart as the time I expect to start working with a task, duration as the time I expect to actually use on the task, and due as the date when the task really should be completed.
+According to the RFC, either due or duration should be given, never both of them.  A dtstart and a duration should be considered as equivalent with a dtstart and a due date (where due = dtstart + duration).  I find that a bit sad, I'd like to use dtstart as the time I expect to start working with a task, duration as the time I expect to actually use on the task, and due as the date when the task really should be completed.  Calendar-cli does not support the duration field as of today.
+
+Sometimes one gets overwhelmed, maybe one gets a week or two behind the schedule due to external circumstances.  calendar-cli supports operations like "add one week to all tasks that haven't been done yet".  As of 2015 the default due date was one week in the future.  It has been changed to one year in the future.  Probably the best practice is to keep the due date unset unless the task has a hard due date.
+
+It's also possible to set a priority field.
+
+The default sorting is:
+
+* overdue tasks at the very top.
+* tasks that have passed dtstart above tasks that haven't passed dtstart
+* within each group, sort by due-date
+* if equal due-date, sort by dtstart
+* if equal due-date and dtstart, sort by priority
+
+My usage pattern so far:
+
+* Skip using the priority field.
+* Try to handle or postpone or delete tasks that are overdue immediately, we shouldn't have any overdue tasks (oups, this didn't work out very well, as the default due date was 7 days in the future).
+* On a daily basis, go through all uncategorized tasks.  All tasks should have at least one category set.  I typically do this while sitting on the metro in the morning.
+* On a daily basis, look through all tasks that have passed the dtstart timestamp.  I'm considering when/how to do those tasks and (re)consider what is most urgent.  It's important to keep this list short (or it gets unwieldy, demotivating, etc), so I procrastinate everything I know I won't get done during the next few days.  I move the dtstart  timestamp to some future date - depending on my capacity, the importance of the tasks, etc, I add either some few days, weeks, months or years to it.
+* Whenever I think of something that needs to be done, I add it on the task list, immediately, from the telephone.
+* I've been using the timestamps to try to prioritize the tasks.
+* Whenever I'm at some specific location or doing some specific work, I'll filter the task list by category.
+
+I believe my approach of using timestamps rather than the priority field makes some sense; by looking through the "task list for this week" on a daily basis, and adding some weeks to those I know I won't be able to do anytime soon, the task list is always being circulated, there are no tasks that really gets forgotten.
 
+Task lists tend to always grow, at some point it's important to realize ... "Those tasks are just not important enough ... I'll probably never get done with those tasks", and simply delete them.
diff --git a/calendar-cli.py b/calendar-cli.py
index 4c349b8..865f740 100755
--- a/calendar-cli.py
+++ b/calendar-cli.py
@@ -24,15 +24,17 @@ import tzlocal
 import time
 from datetime import datetime, timedelta, date
 import dateutil.parser
-from icalendar import Calendar,Event,Todo
+from dateutil.rrule import rrulestr
+from icalendar import Calendar,Event,Todo,Journal,Alarm
 import caldav
 import uuid
 import json
 import os
 import logging
 import sys
+import re
 
-__version__ = "0.10"
+__version__ = "0.11.0-dev"
 __author__ = "Tobias Brox"
 __author_short__ = "tobixen"
 __copyright__ = "Copyright 2013-2016, Tobias Brox"
@@ -45,14 +47,30 @@ __status__ = "Development"
 __product__ = "calendar-cli"
 __description__ = "high-level cli against caldav servers"
 
-def _force_datetime(t):
+def _force_datetime(t, args):
     """
     date objects cannot be compared with timestamp objects, neither in python2 nor python3.  Silly.
+    also, objects with time zone info cannot be compared with timestamps without time zone info.
+    and both datetime.now() and datetime.utcnow() seems to be without those bits.  Silly.
     """
     if type(t) == date:
-        return datetime(t.year, t.month, t.day)
+        t = datetime(t.year, t.month, t.day)
+    if t.tzinfo is None:
+        return t.replace(tzinfo=_tz(args))
+    return t
+
+def _now():
+    """
+    python datetime is ... crap!
+    """
+    return datetime.utcnow().replace(tzinfo=pytz.utc)
+
+def _tz(args):
+    if args.timezone:
+        return pytz.timezone(args.timezone)
     else:
-        return t
+        return tzlocal.get_localzone()
+
 
 ## global constant
 ## (todo: this doesn't really work out that well, leap seconds/days are not considered, and we're missing the month unit)
@@ -63,6 +81,8 @@ time_units = {
 
 vtodo_txt_one = ['location', 'description', 'geo', 'organizer', 'summary']
 vtodo_txt_many = ['categories', 'comment', 'contact', 'resources']
+vcal_txt_one = ['location', 'description']
+vcal_txt_many = []
 
 def niy(*args, **kwargs):
     if 'feature' in kwargs:
@@ -70,8 +90,22 @@ def niy(*args, **kwargs):
     raise NotImplementedError
 
 def caldav_connect(args):
+    ## args.ssl_verify_cert is a string and can be a path or 'yes'/'no'.
+    ## the library expects a path or a boolean.
+    ## Translate 'yes' and 'no' to True and False, or pass the raw string:
+    ssl_verify_cert = {
+        'yes': True,
+        'no': False
+    }.get(args.ssl_verify_cert, args.ssl_verify_cert)
     # Create the account
-    return caldav.DAVClient(url=args.caldav_url, username=args.caldav_user, password=args.caldav_pass)
+    return caldav.DAVClient(url=args.caldav_url, username=args.caldav_user, password=args.caldav_pass, ssl_verify_cert=ssl_verify_cert, proxy=args.caldav_proxy)
+
+def parse_time_delta(delta_string):
+    # TODO: handle bad strings more gracefully
+    if len(delta_string) < 2 or delta_string[-1].lower() not in time_units:
+        raise ValueError("Invalid time delta: %s" % delta_string)
+    num = int(delta_string[:-1])
+    return timedelta(0, num*time_units[delta_string[-1].lower()])
 
 def find_calendar(caldav_conn, args):
     if args.calendar_url:
@@ -171,7 +205,7 @@ def interactive_config(args, config, remaining_argv):
     if not section in config:
         config[section] = {}
 
-    for config_key in ('caldav_url', 'caldav_user', 'caldav_pass', 'language', 'timezone', 'inherits'):
+    for config_key in ('caldav_url', 'caldav_user', 'caldav_pass', 'caldav_proxy', 'ssl_verify_cert', 'language', 'timezone', 'inherits'):
         print("Config option %s - old value: %s" % (config_key, config[section].get(config_key, '(None)')))
         value = raw_input("Enter new value (or just enter to keep the old): ")
         if value:
@@ -211,6 +245,13 @@ def interactive_config(args, config, remaining_argv):
     if args.config_section == 'default' and section != 'default':
         config['default'] = config[section]
     return config
+
+def create_alarm(message, relative_timedelta):
+    alarm = Alarm()
+    alarm.add('ACTION', 'DISPLAY')
+    alarm.add('DESCRIPTION', message)
+    alarm.add('TRIGGER', relative_timedelta, parameters={'VALUE':'DURATION'})
+    return alarm
     
 def calendar_add(caldav_conn, args):
     cal = Calendar()
@@ -251,10 +292,16 @@ def calendar_add(caldav_conn, args):
         event.add('dtend', dtstart + timedelta(0,event_duration_secs))
     ## TODO: what does the cryptic comment here really mean, and why was the dtstamp commented out?  dtstamp is required according to the RFC.
     ## not really correct, and it breaks i.e. with google calendar
-    event.add('dtstamp', datetime.now())
+    event.add('dtstamp', _now())
     ## maybe we should generate some uid?
     uid = uuid.uuid1()
     event.add('uid', str(uid))
+    for attr in vcal_txt_one + vcal_txt_many:
+        if attr == 'summary':
+            continue
+        val = getattr(args, 'set_'+attr)
+        if val:
+            event.add(attr, val)
     event.add('summary', ' '.join(args.summary))
     cal.add_component(event)
     _calendar_addics(caldav_conn, cal.to_ical(), uid, args)
@@ -263,11 +310,7 @@ def calendar_add(caldav_conn, args):
 def calendar_delete(caldav_conn, args):
     cal = find_calendar(caldav_conn, args)
     if args.event_uid:
-        ## TODO: backwards compatibility hack, and/or caldav API in flux hack.  Should go away at some point.
-        if hasattr(cal, 'object_by_uid'):
-            event = cal.object_by_uid(args.event_uid)
-        else:
-            event = cal.event_by_uid(args.event_uid)
+        event = cal.event_by_uid(args.event_uid)
     elif args.event_url:
         event = cal.event_by_url(args.event_url)
     elif args.event_timestamp:
@@ -288,6 +331,24 @@ def calendar_delete(caldav_conn, args):
         raise ValueError("Event deletion failed: either uid, url or timestamp is needed")
     event.delete()
 
+def journal_add(caldav_conn, args):
+    ## TODO: copied from todo_add, should probably be consolidated
+    cal = Calendar()
+    cal.add('prodid', '-//{author_short}//{product}//{language}'.format(author_short=__author_short__, product=__product__, language=args.language))
+    cal.add('version', '2.0')
+    journal = Journal()
+    ## TODO: what does the cryptic comment here really mean, and why was the dtstamp commented out?  dtstamp is required according to the RFC.
+    ## TODO: (cryptic old comment:) not really correct, and it breaks i.e. with google calendar
+    journal.add('dtstamp', datetime.now())
+    journal.add('dtstart', date.today())
+    journal.add('summary', ' '.join(args.summaryline))
+    uid = uuid.uuid1()
+    journal.add('uid', str(uid))
+    cal.add_component(journal)
+    _calendar_addics(caldav_conn, cal.to_ical(), uid, args)
+    print("Added journal item with uid=%s" % uid)
+    ## FULL STOP - should do some major refactoring before doing more work here!
+    
 def todo_add(caldav_conn, args):
     ## TODO: copied from calendar_add, should probably be consolidated
     if args.icalendar or args.nocaldav:
@@ -300,17 +361,15 @@ def todo_add(caldav_conn, args):
     cal.add('prodid', '-//{author_short}//{product}//{language}'.format(author_short=__author_short__, product=__product__, language=args.language))
     cal.add('version', '2.0')
     todo = Todo()
-    ## TODO: what does the cryptic comment here really mean, and why was the dtstamp commented out?  dtstamp is required according to the RFC.
-    ## TODO: (cryptic old comment:) not really correct, and it breaks i.e. with google calendar
-    todo.add('dtstamp', datetime.now())
+    todo.add('dtstamp', _now())
 
-    for arg in ('set_due', 'set_dtstart'):
-        if getattr(args, arg):
-            if type(getattr(args, arg)) == str:
-                val = dateutil.parser.parse(getattr(args, arg))
+    for setarg in ('due', 'dtstart'):
+        if getattr(args, 'set_'+setarg):
+            if type(getattr(args, 'set_'+setarg)) == str:
+                val = dateutil.parser.parse(getattr(args, 'set_'+setarg))
             else:
-                val = getattr(args, arg)
-        todo.add(arg, val)
+                val = getattr(args, 'set_'+setarg)
+        todo.add(setarg, val)
     todo.add('uid', str(uid))
     todo.add('summary', ' '.join(args.summaryline))
     todo.add('status', 'NEEDS-ACTION')
@@ -323,12 +382,26 @@ def todo_add(caldav_conn, args):
             rt.value = str(uid)
             t.save()
     
-    for attr in vtodo_txt_one + vtodo_txt_many:
+    for attr in vtodo_txt_one:
         if attr == 'summary':
             continue
         val = getattr(args, 'set_'+attr)
         if val:
             todo.add(attr, val)
+    ## TODO: this doesn't currently work quite the way we'd like it to
+    ## work (it adds to lines to the ical, and vobject cares only
+    ## about one of them), and if we do get it to work, we'd like to
+    ## refactor and get the same logic in the edit-function
+    for attr in vtodo_txt_many:
+        val = getattr(args, 'set_'+attr)
+        if val:
+            vals = val.split(',')
+            todo.add(attr, vals)
+
+    if args.alarm is not None:
+        alarm = create_alarm(' '.join(args.summaryline), parse_time_delta(args.alarm))
+        todo.add_component(alarm)
+
     cal.add_component(todo)
     _calendar_addics(caldav_conn, cal.to_ical(), uid, args)
     print("Added todo item with uid=%s" % uid)
@@ -343,7 +416,7 @@ def calendar_agenda(caldav_conn, args):
     if args.from_time:
         dtstart = dateutil.parser.parse(args.from_time)
     else:
-        dtstart = datetime.now()
+        dtstart = _now()
     if args.to_time:
         dtend = dateutil.parser.parse(args.to_time)
     elif args.agenda_mins:
@@ -369,11 +442,11 @@ def calendar_agenda(caldav_conn, args):
             else:
                 raise Exception("Panic")
             for event in events__:
-                    dtstart = event.dtstart.value if hasattr(event, 'dtstart') else datetime.now()
+                    dtstart = event.dtstart.value if hasattr(event, 'dtstart') else _now()
                     if not isinstance(dtstart, datetime):
                         dtstart = datetime(dtstart.year, dtstart.month, dtstart.day)
                     if not dtstart.tzinfo:
-                        dtstart = args.timezone.localize(dtstart)
+                        dtstart = _tz(args).localize(dtstart)
                     events.append({'dtstart': dtstart, 'instance': event})
         events.sort(lambda a,b: cmp(a['dtstart'], b['dtstart']))
         for event in events:
@@ -385,7 +458,7 @@ def calendar_agenda(caldav_conn, args):
                     break
             event['uid'] = event['instance'].uid.value if hasattr(event['instance'], 'uid') else '<no uid>'
             ## TODO: this will probably break and is probably moot on python3?
-            if hasattr(event['summary'], 'encode'):
+            if isinstance(event['summary'], unicode):
                 event['summary'] = event['summary'].encode('utf-8')
             print(args.event_template.format(**event))
 
@@ -398,12 +471,34 @@ def todo_select(caldav_conn, args):
         ## TODO: we're fetching everything from the server, and then doing the filtering here.  It would be better to let the server do the filtering, though that requires library modifications.
         ## TODO: current release of the caldav library doesn't support the multi-key sort_keys attribute.  The try-except construct should be removed at some point in the future, when caldav 0.5 is released.
         try:
-            tasks = find_calendar(caldav_conn, args).todos(sort_keys=('dtstart', 'due', 'priority'))
+            tasks = find_calendar(caldav_conn, args).todos(sort_keys=('isnt_overdue', 'hasnt_started', 'due', 'dtstart', 'priority'))
         except:
             tasks = find_calendar(caldav_conn, args).todos()
     for attr in vtodo_txt_one + vtodo_txt_many: ## TODO: now we have _exact_ match on items in the the array attributes, and substring match on items that cannot be duplicated.  Does that make sense?  Probably not.
         if getattr(args, attr):
             tasks = [x for x in tasks if hasattr(x.instance.vtodo, attr) and getattr(args, attr) in getattr(x.instance.vtodo, attr).value]
+        if getattr(args, 'no'+attr):
+            tasks = [x for x in tasks if not hasattr(x.instance.vtodo, attr)]
+    if args.overdue:
+        tasks = [x for x in tasks if hasattr(x.instance.vtodo, 'due') and _force_datetime(x.instance.vtodo.due.value, args) < _force_datetime(datetime.now(), args)]
+    if args.hide_future:
+        tasks = [x for x in tasks if not(hasattr(x.instance.vtodo, 'dtstart') and _force_datetime(x.instance.vtodo.dtstart.value, args) > _force_datetime(datetime.now(), args))]
+    if args.hide_parents or args.hide_children:
+        tasks_by_uid = {}
+        for task in tasks:
+            tasks_by_uid[task.instance.vtodo.uid.value] = task
+        for task in tasks:
+            if hasattr(task.instance.vtodo, 'related_to'):
+                uid = task.instance.vtodo.uid.value
+                rel_uid = task.instance.vtodo.related_to.value
+                rel_type = task.instance.vtodo.related_to.params.get('RELTYPE', 'PARENT')
+                if ((rel_type == 'CHILD' and args.hide_parents) or (rel_type == 'PARENT' and args.hide_children)) and \
+                   rel_uid in tasks_by_uid and uid in tasks_by_uid:
+                    del tasks_by_uid[uid]
+                if ((rel_type == 'PARENT' and args.hide_parents) or (rel_type == 'CHILD' and args.hide_children)) and \
+                   rel_uid in tasks_by_uid:
+                    del tasks_by_uid[rel_uid]
+        tasks = [x for x in tasks if x.instance.vtodo.uid.value in tasks_by_uid]
     if args.top+args.limit:
         tasks = tasks[args.offset+args.offsetn:args.top+args.limit+args.offset+args.offsetn]
     elif args.offset+args.offsetn:
@@ -445,7 +540,7 @@ def todo_postpone(caldav_conn, args):
     if args.until.startswith('+'):
         rel_skew = timedelta(seconds=int(args.until[1:-1])*time_units[args.until[-1]])
     elif args.until.startswith('in'):
-        new_ts = datetime.now()+timedelta(seconds=int(args.until[2:-1])*time_units[args.until[-1]])
+        new_ts = _now()+timedelta(seconds=int(args.until[2:-1])*time_units[args.until[-1]])
     else:
         new_ts = dateutil.parser.parse(args.until)
         if not new_ts.time():
@@ -496,16 +591,18 @@ def todo_list(caldav_conn, args):
         for task in tasks:
             t = {'instance': task}
             t['dtstart'] = task.instance.vtodo.dtstart.value if hasattr(task.instance.vtodo,'dtstart') else date.today()
-            t['dtstart_passed_mark'] = '!' if _force_datetime(t['dtstart']) <= datetime.now() else ' '
-            t['due'] = task.instance.vtodo.due.value if hasattr(task.instance.vtodo,'due') else date.today()+timedelta(365)
-            t['due_passed_mark'] = '!' if _force_datetime(t['due']) < datetime.now() else ' '
+            t['dtstart_passed_mark'] = '!' if _force_datetime(t['dtstart'], args) <= _now() else ' '
+            t['due'] = task.instance.vtodo.due.value if hasattr(task.instance.vtodo,'due') else date.today()+timedelta(args.default_due)
+            t['due_passed_mark'] = '!' if _force_datetime(t['due'], args) < _now() else ' '
+            for timeattr in ('dtstart', 'due'):
+                t[timeattr] = t[timeattr].strftime(args.timestamp_format)
             for summary_attr in ('summary', 'location', 'description', 'url', 'uid'):
                 if hasattr(task.instance.vtodo, summary_attr):
                     t['summary'] = getattr(task.instance.vtodo, summary_attr).value
                     break
             t['uid'] = task.instance.vtodo.uid.value
             ## TODO: this will probably break and is probably moot on python3?
-            if hasattr(t['summary'], 'encode'):
+            if hasattr(t['summary'], 'encode') and isinstance(t['summary'], unicode):
                 t['summary'] = t['summary'].encode('utf-8')
             print(args.todo_template.format(**t))
 
@@ -514,7 +611,43 @@ def todo_complete(caldav_conn, args):
         raise ValueError("No caldav connection, aborting")
     tasks = todo_select(caldav_conn, args)
     for task in tasks:
+        if hasattr(task.instance.vtodo, 'rrule'):
+            rrule = rrulestr(task.instance.vtodo.rrule.value)
+            try:
+                next = rrule.after(datetime.now())
+            except TypeError: ## pesky problem with comparition of timestamps with and without tzinfo
+                next = rrule.after(datetime.now(tz=tzlocal.get_localzone()))
+            if next:
+                ## new_task is to be completed and we keep the original task open
+                completed_task = task.copy()
+                remaining_task = task
+
+                ## the remaining task should have recurrence id set to next start time, and range THISANDFUTURE
+                if hasattr(remaining_task.instance.vtodo, 'recurrence_id'):
+                    del remaining_task.instance.vtodo.recurrence_id
+                remaining_task.instance.vtodo.add('recurrence-id')
+                remaining_task.instance.vtodo.recurrence_id.value = next ## TODO: should be same type as dtstart (date or datetime)
+                remaining_task.instance.vtodo.dtstart.value = next ## TODO: should be same type as dtstart (date or datetime)
+                remaining_task.instance.vtodo.recurrence_id.params['RANGE'] = [ 'THISANDFUTURE' ]
+                remaining_task.instance.vtodo.rrule
+                remaining_task.save()
+
+                ## the completed task should have recurrence id set to current time
+                ## count in rrule should decrease
+                if hasattr(completed_task.instance.vtodo, 'recurrence_id'):
+                    del completed_task.instance.vtodo.recurrence_id
+                completed_task.instance.vtodo.add('recurrence-id')
+                completed_task.instance.vtodo.recurrence_id.value = datetime.now()
+                completed_task.instance.vtodo.dtstart.value = datetime.now()
+                count_search = re.search('COUNT=(\d+)', completed_task.instance.vtodo.rrule.value)
+                if count_search:
+                    completed_task.instance.vtodo.rrule.value = re.replace('COUNT=(\d+)', 'COUNT=%d' % int(count_search.group(1))-1)
+                completed_task.complete()
+
+                continue
+
         task.complete()
+            
 
 def todo_delete(caldav_conn, args):
     if args.nocaldav:
@@ -524,11 +657,12 @@ def todo_delete(caldav_conn, args):
         task.delete()
         
 def config_section(config, section='default'):
-    if 'inherits' in config[section]:
+    if section in config and 'inherits' in config[section]:
         ret = config_section(config, config[section]['inherits'])
     else:
         ret = {}
-    ret.update(config[section])
+    if section in config:
+        ret.update(config[section])
     return ret
     
 def main():
@@ -579,6 +713,11 @@ def main():
             return
     else:
         defaults = config_section(config, args.config_section)
+        if not 'ssl_verify_cert' in defaults:
+            defaults['ssl_verify_cert'] = 'yes'
+        if not 'language' in defaults:
+            ## TODO: shouldn't this be lower case?
+            defaults['language'] = 'EN'
 
     # Parse rest of arguments
     # Don't suppress add_help here so it will handle -h
@@ -594,10 +733,12 @@ def main():
     parser.add_argument("--nocaldav", help="Do not connect to CalDAV server, but read/write icalendar format from stdin/stdout", action="store_true")
     parser.add_argument("--icalendar", help="Read/write icalendar format from stdin/stdout", action="store_true")
     parser.add_argument("--timezone", help="Timezone to use")
-    parser.add_argument('--language', help="language used", default="EN")
+    parser.add_argument('--language', help="language used")
     parser.add_argument("--caldav-url", help="Full URL to the caldav server", metavar="URL")
     parser.add_argument("--caldav-user", help="username to log into the caldav server", metavar="USER")
     parser.add_argument("--caldav-pass", help="password to log into the caldav server", metavar="PASS")
+    parser.add_argument("--caldav-proxy", help="HTTP proxy server to use (if any)")
+    parser.add_argument("--ssl-verify-cert", help="verification of the SSL cert - 'yes' to use the OS-provided CA-bundle, 'no' to trust any cert and the path to a CA-bundle")
     parser.add_argument("--debug-logging", help="turn on debug logging", action="store_true")
     parser.add_argument("--calendar-url", help="URL for calendar to be used (may be absolute or relative to caldav URL, or just the name of the calendar)")
 
@@ -612,28 +753,40 @@ def main():
     todo_parser.add_argument('--offsetn', type=int, default=0)
     todo_parser.add_argument('--limit', type=int, default=0)
     todo_parser.add_argument('--todo-uid')
+    todo_parser.add_argument('--hide-parents', help='Hide the parent if you need to work on children tasks first (parent task depends on children tasks to be done first)', action='store_true')
+    todo_parser.add_argument('--hide-children', help='Hide the parent if you need to work on children tasks first (parent task depends on children tasks to be done first)', action='store_true')
+    todo_parser.add_argument('--overdue', help='Only show overdue tasks', action='store_true')
+    todo_parser.add_argument('--hide-future', help='Hide events with future dtstart', action='store_true')
 
     for attr in vtodo_txt_one + vtodo_txt_many:
         todo_parser.add_argument('--'+attr, help="for filtering tasks")
-    
+
+    for attr in vtodo_txt_one + vtodo_txt_many:
+        todo_parser.add_argument('--no'+attr, help="for filtering tasks", action='store_true')
+        
     #todo_parser.add_argument('--priority', ....) 
     #todo_parser.add_argument('--sort-by', ....)
     #todo_parser.add_argument('--due-before', ....)
     todo_subparsers = todo_parser.add_subparsers(title='tasks subcommand')
     todo_add_parser = todo_subparsers.add_parser('add')
     todo_add_parser.add_argument('summaryline', nargs='+')
-    todo_add_parser.add_argument('--set-due', default=date.today()+timedelta(7))
+    todo_add_parser.add_argument('--set-due', default=date.today()+timedelta(365))
     todo_add_parser.add_argument('--set-dtstart', default=date.today()+timedelta(1))
     todo_add_parser.add_argument('--is-child', help="the new task is a child-task of the selected task(s)", action='store_true')
     for attr in vtodo_txt_one + vtodo_txt_many:
         if attr != 'summary':
             todo_add_parser.add_argument('--set-'+attr, help="Set "+attr)
+    # TODO: we probably want to be able to set or delete alarms in other situations, yes?  generalize?
+    todo_add_parser.add_argument('--alarm', metavar='DURATION_BEFORE',
+        help="specifies a time at which a reminder should be presented for this task, " \
+             "relative to the start time of the task (as a timestamp delta)")
     todo_add_parser.set_defaults(func=todo_add)
     
     todo_list_parser = todo_subparsers.add_parser('list')
     todo_list_parser.add_argument('--todo-template', help="Template for printing out the event", default="{dtstart}{dtstart_passed_mark} {due}{due_passed_mark} {summary}")
-    todo_list_parser.add_argument('--default-due', help="Default number of days from a task is submitted until it's considered due", default=14)
+    todo_list_parser.add_argument('--default-due', help="Default number of days from a task is submitted until it's considered due", type=int, default=365)
     todo_list_parser.add_argument('--list-categories', help="Instead of listing the todo-items, list the unique categories used", action='store_true')
+    todo_list_parser.add_argument('--timestamp-format', help="strftime-style format string for the output timestamps", default="%F (%a)")
     todo_list_parser.set_defaults(func=todo_list)
 
     todo_edit_parser = todo_subparsers.add_parser('edit')
@@ -654,7 +807,14 @@ def main():
 
     todo_delete_parser = todo_subparsers.add_parser('delete')
     todo_delete_parser.set_defaults(func=todo_delete)
-    
+
+    ## journal
+    journal_parser = subparsers.add_parser('journal')
+    journal_subparsers = journal_parser.add_subparsers(title='tasks subcommand')
+    journal_add_parser = journal_subparsers.add_parser('add')
+    journal_add_parser.add_argument('summaryline', nargs='+')
+    journal_add_parser.set_defaults(func=journal_add)
+
     calendar_parser = subparsers.add_parser('calendar')
     calendar_subparsers = calendar_parser.add_subparsers(title='cal subcommand')
     calendar_add_parser = calendar_subparsers.add_parser('add')
@@ -663,6 +823,9 @@ def main():
     calendar_add_parser.set_defaults(func=calendar_add)
     calendar_add_parser.add_argument('--whole-day', help='Whole-day event', action='store_true', default=False)
 
+    for attr in vcal_txt_one + vcal_txt_many:
+        calendar_add_parser.add_argument('--set-'+attr, help='Set '+attr)
+
     calendar_addics_parser = calendar_subparsers.add_parser('addics')
     calendar_addics_parser.add_argument('--file', help="ICS file to upload", default='-')
     calendar_addics_parser.set_defaults(func=calendar_addics)
@@ -684,13 +847,10 @@ def main():
 
     args = parser.parse_args(remaining_argv)
 
-    if args.timezone:
-        args.timezone = pytz.timezone(args.timezone)
-    else:
-        args.timezone = tzlocal.get_localzone()
-        
     if not args.nocaldav:
         caldav_conn = caldav_connect(args)
+    else:
+        caldav_conn = None
 
     ret = args.func(caldav_conn, args)
 
diff --git a/tests/script_test.sh b/tests/script_test.sh
index 375acdb..1636478 100755
--- a/tests/script_test.sh
+++ b/tests/script_test.sh
@@ -70,7 +70,7 @@ calendar_cli todo --categories scripttest list
 [ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found the todo item we just added and nothing more"
 
 echo "## Editing the task"
-calendar_cli todo --categories scripttest edit --set-comment "editing" --add-categories "scripttest2"
+calendar_cli todo --categories scripttest edit --set-summary "editing" --add-categories "scripttest2"
 
 echo "## Verifying that the edits got through"
 calendar_cli todo --categories scripttest list
@@ -84,6 +84,29 @@ echo "## Complete the task"
 calendar_cli todo --categories scripttest complete
 calendar_cli todo --categories scripttest list
 [ -z "$output" ] && echo "## OK: todo-item is done"
-calendar_cli todo --todo-uid $todouid1 delete
+calendar_cli todo --todo-uid $uidtodo1 delete
+
+## parent-child relationships
+echo "## Going to add three todo-items with children/parent relationships"
+calendar_cli todo add --set-categories scripttest "this is a grandparent"
+uidtodo2=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories=scripttest add --set-categories scripttest --is-child "this is a parent and a child"
+uidtodo3=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories=scripttest add --set-categories scripttest --is-child "this task has two parents"
+uidtodo4=$(echo $output | perl -ne '/uid=(.*)$/ && print $1')
+calendar_cli todo --categories scripttest list
+[ $(echo "$output" | wc -l) == 3 ] && echo "## OK: found three tasks"
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+echo "## Going to complete the children task"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ $(echo "$output" | wc -l) == 1 ] && echo "## OK: found only one task now"
+calendar_cli todo --hide-parents --categories scripttest complete
+calendar_cli todo --hide-parents --categories scripttest list
+[ -z "$output" ] && echo "## OK: found no tasks now"
 
 echo "## all tests completed"