encoding problems on Mac OSX default installation - provide more helpful error messages or fall back on en_US (ASCII)?

[zsoldosp]

I've just helped a non-developer friend on Mac setup & get started with blogofile (0.8.1b with python 2.7.1) , and we have run into numerous exceptions, that all boiled down to an unset locale issue (similar to http://www.selenic.com/pipermail/mercurial/2007-October/015296.html)

Our actions & then the error messages a sample blogofile build command gave us are included below. I would suggest either displaying the following message instead of the error messages when the returned locale is the default utf-8:

"""Incomplete terminal setup. Please fix this by the issuing the following commands:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8"""

or automatically falling back to the en_US locale and maybe displaying a warning (like when there is a post without yaml) that locale is not specified, falling back to en_US (and I suggest en_US because then we ran into issues with the terminal not being set up to handle utf-8 so an ascii locale is a safe choice)

----- stack traces in order ---

$ blogofile build
ERROR:blogofile.filter:Cannot load filter: rst_template
Traceback (most recent call last):
  File "/usr/local/bin/blogofile", line 8, in <module>
    load_entry_point('Blogofile==0.8b1', 'console_scripts', 'blogofile')()
  File "/Library/Python/2.7/site-packages/blogofile/main.py", line 58, in main
    args.func(args)
  File "/Library/Python/2.7/site-packages/blogofile/main.py", line 382, in do_build
    config.init_interactive(args)
  File "/Library/Python/2.7/site-packages/blogofile/config.py", line 47, in init_interactive
    _load_config("_config.py")
  File "/Library/Python/2.7/site-packages/blogofile/config.py", line 74, in _load_config
    plugin.load_plugins()
  File "/Library/Python/2.7/site-packages/blogofile/plugin.py", line 85, in load_plugins
    directory=os.path.join(plugin_dir, "site_src", "_filters"))
  File "/Library/Python/2.7/site-packages/blogofile/filter.py", line 72, in preload_filters
    load_filter(fn[:-3], module_path=p, namespace=namespace)
  File "/Library/Python/2.7/site-packages/blogofile/filter.py", line 132, in load_filter
    "{0}_{1}".format(name, uuid.uuid4()), module_path)
  File "/Library/Python/2.7/site-packages/blogofile_blog/site_src/_filters/rst_template.py", line 2, in <module>
    import docutils.core
  File "/Library/Python/2.7/site-packages/docutils/core.py", line 20, in <module>
    from docutils import frontend, io, utils, readers, writers
  File "/Library/Python/2.7/site-packages/docutils/frontend.py", line 39, in <module>
    import docutils.utils
  File "/Library/Python/2.7/site-packages/docutils/utils/__init__.py", line 19, in <module>
    from docutils.io import FileOutput
  File "/Library/Python/2.7/site-packages/docutils/io.py", line 18, in <module>
    from docutils.error_reporting import locale_encoding, ErrorString, ErrorOutput
  File "/Library/Python/2.7/site-packages/docutils/error_reporting.py", line 47, in <module>
    locale_encoding = locale.getlocale()[1] or locale.getdefaultlocale()[1]
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 515, in getlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 428, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
$ export LC_ALL=hu_HU.UTF-8
$ export LANG=hu_HU.UTF-8
$ blogofile build
ERROR:blogofile.template:Error rendering template: rss.mako


Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/blogofile/template.py", line 157, in render
    rendered = self.mako_template.render(**self)
  File "/Library/Python/2.7/site-packages/mako/template.py", line 412, in render
    return runtime._render(self, self.callable_, args, data)
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 766, in _render
    **_kwargs_for_callable(callable_, data))
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 798, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 824, in _exec_template
    callable_(context, *args, **kwargs)
  File "_templates/blog/rss.mako", line 13, in render_body
    <pubDate>${datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S GMT")}</pubDate>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

ERROR:blogofile:Fatal build error occured, calling bf.config.build_exception()
Traceback (most recent call last):
  File "/usr/local/bin/blogofile", line 8, in <module>
    load_entry_point('Blogofile==0.8b1', 'console_scripts', 'blogofile')()
  File "/Library/Python/2.7/site-packages/blogofile/main.py", line 58, in main
    args.func(args)
  File "/Library/Python/2.7/site-packages/blogofile/main.py", line 388, in do_build
    writer.write_site()
  File "/Library/Python/2.7/site-packages/blogofile/writer.py", line 49, in write_site
    self.__run_controllers()
  File "/Library/Python/2.7/site-packages/blogofile/writer.py", line 162, in __run_controllers
    controller.run_all(namespaces)
  File "/Library/Python/2.7/site-packages/blogofile/controller.py", line 229, in run_all
    c.mod.run()
  File "/Library/Python/2.7/site-packages/blogofile_blog/site_src/_controllers/blog/__init__.py", line 83, in run
    categories.run()
  File "/Library/Python/2.7/site-packages/blogofile_blog/site_src/_controllers/blog/categories.py", line 11, in run
    write_categories()
  File "/Library/Python/2.7/site-packages/blogofile_blog/site_src/_controllers/blog/categories.py", line 40, in write_categories
    feed.write_feed(category_posts, rss_path, "rss.mako")
  File "/Library/Python/2.7/site-packages/blogofile_blog/site_src/_controllers/blog/feed.py", line 18, in write_feed
    tools.materialize_template(template, path, env)
  File "/Library/Python/2.7/site-packages/blogofile/plugin.py", line 160, in materialize_template
    lookup=self.template_lookup, caller=self.module)
  File "/Library/Python/2.7/site-packages/blogofile/template.py", line 388, in materialize_template
    template.render(location)
  File "/Library/Python/2.7/site-packages/blogofile/template.py", line 157, in render
    rendered = self.mako_template.render(**self)
  File "/Library/Python/2.7/site-packages/mako/template.py", line 412, in render
    return runtime._render(self, self.callable_, args, data)
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 766, in _render
    **_kwargs_for_callable(callable_, data))
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 798, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/Library/Python/2.7/site-packages/mako/runtime.py", line 824, in _exec_template
    callable_(context, *args, **kwargs)
  File "rss_mako", line 38, in render_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8
$ blogofile build # success!

just to document, a typo resulted in another unhelpful for the uninitiated error message, though this may be just icing on the cake to handle more gracefully

$ export LC_ALL=en_EN.UTF-8
 blogofile build
Traceback (most recent call last):
  File "/usr/local/bin/blogofile", line 8, in <module>
    load_entry_point('Blogofile==0.8b1', 'console_scripts', 'blogofile')()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 318, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2221, in load_entry_point
    return ep.load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/Library/Python/2.7/site-packages/blogofile/main.py", line 29, in <module>
    locale.setlocale(locale.LC_ALL, '')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 531, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

[douglatornell]

Thanks for the detailed issue report.

I like the idea of falling back to a default locale like en_CA (or en_US, if you insist :-) and displaying a warning.

[iuriguilherme]

Same issue of #138, #141, #147, #148 and #152