options: Parse config file before opening working directory

Right now, the `config` file is read *after* the configuration working directory is moved to in the software. However one configuration option `lightning-dir` settable in the `config` file sets this working directory. As the directory is already opened (which defaults to `$HOME/.lightning`) before the configuration is read, the configured directory will not be used.

This patch parses the configuration file before opening the working directory, fixing this bug.

[ Update CHANGELOG.md and man pages -- RR ]

CHANGELOG.md

@@ -19,6 +19,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 - Protocol: `channel_update` sent to disable channel only if we reject an HTLC.
 - Protocol: we don't send redundant `node_announcement` on every new channel.
+- Config: config file can override `lightning-dir` (makes sense with `--conf`).
+- Config: `--conf` option is now relative to current directory, not `lightning-dir`.
 
 ### Deprecated
 

doc/lightningd-config.5

@@ -2,12 +2,12 @@
 .\"     Title: lightningd-config
 .\"    Author: [see the "AUTHOR" section]
 .\" Generator: DocBook XSL Stylesheets v1.79.1 <http://docbook.sf.net/>
-.\"      Date: 09/07/2018
+.\"      Date: 10/12/2018
 .\"    Manual: \ \&
 .\"    Source: \ \&
 .\"  Language: English
 .\"
-.TH "LIGHTNINGD\-CONFIG" "5" "09/07/2018" "\ \&" "\ \&"
+.TH "LIGHTNINGD\-CONFIG" "5" "10/12/2018" "\ \&" "\ \&"
 .\" -----------------------------------------------------------------
 .\" * Define some portability stuff
 .\" -----------------------------------------------------------------
@@ -34,7 +34,7 @@ lightningd-config \- Lightning daemon configuration file
 \fB~/\&.lightningd/config\fR
 .SH "DESCRIPTION"
 .sp
-lightningd(8) reads a configuration file called \fIconfig\fR, if it exists, when it starts up\&. The location of this file defaults to \fB\&.lightning\fR in the home directory, but can be overridden by the \fI\-\-lightning\-dir\fR option on the lightningd(8) command line\&.
+When lightningd(8) starts up, it reads a configuration file\&. By default that is \fIconfig\fR in the \fB\&.lightning\fR subdirectory of the home directory (if it exists), but that can be changes by the \fI\-\-lightning\-dir\fR or \fI\-\-conf\fR options on the lightningd(8) command line\&.
 .sp
 Configuration file options are processed first, then command line options: later options override earlier ones except \fIaddr\fR options which accumulate\&.
 .sp
@@ -131,7 +131,9 @@ Lightning daemon options:
 .PP
 \fBlightning\-dir\fR=\fIDIR\fR
 .RS 4
-Sets the working directory\&. All other files are relative to this\&.
+Sets the working directory\&. All files (except
+\fI\-\-conf\fR
+on the command line) are relative to this\&.
 .RE
 .PP
 \fBpid\-file\fR=\fIPATH\fR
@@ -166,7 +168,13 @@ Run in the background, suppress stdout and stderr\&.
 .PP
 \fBconf\fR=\fIPATH\fR
 .RS 4
-Sets configuration file\&. Relative paths will be prefixed by lightning\-dir location\&. (default: config)
+Sets configuration file (default:
+\fBlightning\-dir\fR/\fIconfig\fR
+)\&. If this is a relative path, it is relative to the starting directory, not
+\fBlightning\-dir\fR
+(unlike other paths)\&.
+\fIPATH\fR
+must exist and be readable (we allow missing files in the default case)\&. Using this inside a configuration file is meaningless\&.
 .RE
 .sp
 Lightning node customization options:

doc/lightningd-config.5.txt

@@ -13,10 +13,11 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-lightningd(8) reads a configuration file called 'config', if it
-exists, when it starts up.  The location of this file defaults to
-*.lightning* in the home directory, but can be overridden by the
-'--lightning-dir' option on the lightningd(8) command line.
+When lightningd(8) starts up, it reads a configuration file.  By
+default that is 'config' in the *.lightning* subdirectory of the home
+directory (if it exists), but that can be changes by the
+'--lightning-dir' or '--conf' options on the lightningd(8) command
+line.
 
 Configuration file options are processed first, then command line
 options: later options override earlier ones except 'addr' options
@@ -93,7 +94,8 @@ Bitcoin control options:
 Lightning daemon options:
 
 *lightning-dir*='DIR'::
-    Sets the working directory. All other files are relative to this.
+    Sets the working directory. All files (except '--conf' on the command
+    line) are relative to this.
 
 *pid-file*='PATH'::
     Specify pid file to write to.
@@ -116,7 +118,11 @@ Lightning daemon options:
     Run in the background, suppress stdout and stderr.
 
 *conf*='PATH'::
-    Sets configuration file. Relative paths will be prefixed by lightning-dir location. (default: config)
+    Sets configuration file (default: *lightning-dir*/'config' ).  If this is
+    a relative path, it is relative to the starting directory, not
+    *lightning-dir* (unlike other paths).  'PATH' must exist and be readable
+    (we allow missing files in the default case).
+    Using this inside a configuration file is meaningless.
 
 Lightning node customization options:
 

lightningd/options.c

@@ -613,17 +613,21 @@ static void opt_parse_from_config(struct lightningd *ld)
 	char **all_args; /*For each line: either argument string or NULL*/
 	char *argv[3];
 	int i, argc;
+	char *filename;
+
+	if (ld->config_filename != NULL)
+		filename = ld->config_filename;
+	else
+		filename = path_join(tmpctx, ld->config_dir, "config");
+
+	contents = grab_file(ld, filename);
 
-	if (ld->config_filename != NULL) {
-		contents = grab_file(ld, ld->config_filename);
-	} else
-		contents = grab_file(ld, "config");
 	/* The default config doesn't have to exist, but if the config was
 	 * specified on the command line it has to exist. */
 	if (!contents) {
 		if ((errno != ENOENT) || (ld->config_filename != NULL))
-			fatal("Opening and reading config: %s",
-			      strerror(errno));
+			fatal("Opening and reading %s: %s",
+			      filename, strerror(errno));
 		/* Now we can set up defaults, since no config file. */
 		setup_default_config(ld);
 		return;
@@ -805,6 +809,9 @@ void handle_opts(struct lightningd *ld, int argc, char *argv[])
 	/* Get any configdir/testnet options first. */
 	opt_early_parse(argc, argv, opt_log_stderr_exit);
 
+	/* Now look for config file */
+	opt_parse_from_config(ld);
+
 	/* Move to config dir, to save ourselves the hassle of path manip. */
 	if (chdir(ld->config_dir) != 0) {
 		log_unusual(ld->log, "Creating configuration directory %s",
@@ -817,9 +824,6 @@ void handle_opts(struct lightningd *ld, int argc, char *argv[])
 			      ld->config_dir, strerror(errno));
 	}
 
-	/* Now look for config file */
-	opt_parse_from_config(ld);
-
 	opt_parse(&argc, argv, opt_log_stderr_exit);
 	if (argc != 1)
 		errx(1, "no arguments accepted");