Needed to change a couple of migrations. The changes are mostly innocuous:
- changing BLOB to TEXT for short_channel_ids which is the correct type
anyway, and sqlite3 treats them the same anyway.
- Use `int` for version since the byte representation is checked by postgres.
- Change anything that is INT, but will be bound to u64 to BIGINT (again
postgres checks these more carefully than sqlite3).
Two migrations were replaced with dummy values, since they are buried deep
enough, and I found no portable way of expressing `strftime()` and `INSERT OR
IGNORE`.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Using a generated identifier with filename and line proved to be brittle since
compilers assign the __LINE__ macro differently on multi-line macro
invocations.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This is dangerous but needed since postgres is not as forgiving about
unsatisfied foreign key constraints even while in a DB transaction.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We used to do some of the setup work in db.c, which is now free of any
sqlite3-specific code. In addition we also switch over to fully qualified DSNs
to specify the location of the wallet.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This avoids having to correlate with listpeers for the most pertinent
information.
This API predates plugins, otherwise we'd have listutxos and listpeers
and this would simply combine them appropriately. Still, it exists so
there's little reason not to make it more friendly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Allow a user to select the utxo set that will be added to a
transaction, via the `utxos` parameter. Optional.
Format for utxos should be of the form ["txid:vout","..."]
Rather than reaching into data structures, let them register their own
callbacks. This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.
Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're about to remove them.
Includes fix to sqlite3_bind_short_channel_id to not assume `id` is a
tal object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now that all the users are migrated to the abstraction layer we can remove the
legacy implementation.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We are about to delete all the `sqlite3`-specific code from `db.c` and this is
one of the last uses of the old interface.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
It's better to let the driver decide when and how to expand. It can then
report the expanded statement back to the dispatch through the
`db_changes_add` function.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We now have a much stronger consistency check from the combination of
transaction wrapping, tal memory leak detection. Tramsaction wrapping ensures
that each statement is executed before the transaction is committed. The
commit is also driven by the `io_loop`, which means that it is no longer
possible for us to have statements outside of transactions and transactions
are guaranteed to commit at the round's end.
By adding the tal-awareness we can also get a much better indication as to
whether we have un-freed statements flying around, which we can test at the
end of the round as well.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This is likely the last part we need to completely encapsulate the part of the
sqlite3 API that we were using. Like the `db_count_changes` call I decided to
pass in the `struct db_stmt` since really they refer to the statement that was
executed and not the db.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These are based on top of the basic column access functions, and act as a
small type-safe wrapper, that also does a bit of validation.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This has a slight side-effect of removing the actual begin and commit
statements from the `db_write` hooks, but they are mostly redundant anyway (no
harm in grouping pre-init statements into one transaction, and we know that
each post-init call is supposed to be wrapped anyway).
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These are used to do one-time initializations and wait for pending statements
before closing.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
I was hoping to get rid of these by using "ON CONFLICT" upserts, however
sqlite3 only started supporting them in version 3.24.0 which is newer than
some of our deployment targets.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This is the first step towards being able to extract information from query
rows. Only the most basic types are exposed, the others will be built on top
of these primitives.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
For some of the query methods in the next step we need to have an idea of
whether the stmt was executed (db_step function) so let's track that
explicitly.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These do not require the ability to iterate over the result, hence they can be
migrated already.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These functions implement the lookup of the query, and the dispatch to the
DB-specific functions that do the actual heavy lifting.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This gets rid of the two parallel execution paths of read-only and write
queries, by explicitly stating with each query whether it is a read-only
query, we only need to remember the ones marked as write queries.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
All drivers will have to reach into it, so put it in a place that is reachable
from the drivers, along with all other definitions.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This is the counterpart of the annotations we did in the last few commits. It
extracts queries, passes them through a driver-specific query rewriter and
dumps them into a driver-specific query-list, along with some metadata to
facilitate processing later on. The generated query list is then registered as
a `db_config` and will be loaded by the driver upon instantiation.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We will soon generalize the DB, so directly reaching into the `struct db`
instance to talk to the sqlite3 connection is bad anyway. This increases
flexibility and allows us to tailor the actual implementation to the
underlying DB.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These two simple macros have a twofold use:
1) They serve as annotations for the query extraction tool to find them when
extracting queries from the C source code.
2) They replace the actual queries with names that can be used to lookup the
queries in a table again, once they have been rewritten into the target SQL dialect.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
`db_select_prepare` was prepending the "SELECT" part in an attempt to limit
its use to read-only statements. This is leads to the queries in the code not
actually being well-formed, which we'll need in a later commit, and was also
resulting in extra allocations. This switches the behavior to just enforce a
"SELECT" prefix being present which allows us to have well-formed queries in
the code again and avoids the extra allocation.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We need to have full DB queries that can be extracted at compile time later in
order to be able to rewrite them in other SQL dialects. In addition we had a
bit of unnecessary code-duplication in db_select and db_select_prepare. Now
the former uses the latter internally.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
These will interfere with our query extraction process later on, and they were
really separating definition from use anyway, so let's expand these field lists.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Christian Decker
Rusty Russell
lisa neigut
trueptolemy