[Request for comment] Addition of sqlite3 client for script authors

[RMerlin]

Primarily intended at the author of advanced scripts and other add-on features in the firmware. Since the sqlite3 library is already there, I could add the sqlite3 user tool to allow scripters to store data in an sqlite3 database.

 

[L&LD]

As a user of the advanced scripts available, I think any addition that makes the job for the developers easier is most welcome.

Particularly when it is a 54KB addition to the firmware size.

 

[Jack Yaz]

This would allow me to move away from rrdtool (removing my dependency on Entware completely for some scripts), and be able to work with databases in a cleaner way to extract/pass to chart.js

So +1 from me!

 

[sinshiva]

Would potentially be useful for Strongswan users wanting to configure hundreds of xauth users lol

https://wiki.strongswan.org/projects/strongswan/wiki/Attrsql

 

[Butterfly Bones]

Yes, please.

 

[FreshJR]

Any chance for database write permissions via WebUI?

 

[thelonelycoder]

Just the other day I was toying with the idea of using sqlite to store Diversions config settings and more.
This would help as it is available before Entware is installed.

 

[RMerlin]

Any chance for database write permissions via WebUI?

A webpage cannot "write" to a server, it's just the output generated by that server's httpd. And implementing callbacks to allow arbitrary writes to the filesystem would be a major security risks, since httpd has to run as root.

 

[Martineau]

Primarily intended at the author of advanced scripts and other add-on features in the firmware. Since the sqlite3 library is already there, I could add the sqlite3 user tool to allow scripters to store data in an sqlite3 database.

Whilst I do exploit creating SQL databases for my internal reporting etc., I believe currently there are only four public scripts? (three published by me) that use the sqlite3 user-space tool to read existing SQL databases, so adding the user-space tool to the firmware may have limited benefit, given most environments install Entware anyway.

..but +1 anyway.

 

[Dabombber]

This would be really nice considering the current lack of any data processing ability (technically cvs would work but ehhhh).

And implementing callbacks to allow arbitrary writes to the filesystem would be a major security risks, since httpd has to run as root.

It could be limited to /www/ext, although I still don't think it's worth adding compared to just using lighttpd/nginx.

 

[Jack Yaz]

This would be really nice considering the current lack of any data processing ability (technically cvs would work but ehhhh).


It could be limited to /www/ext, although I still don't think it's worth adding compared to just using lighttpd/nginx.

Yeah I considered moving from rrd to csv in the short term for chart.js, but a recent commit looks like I might just wait a little bit

 

[RMerlin]

Whilst I do exploit creating SQL databases for my internal reporting etc., I believe currently there are only four public scripts? (three published by me) that use the sqlite3 user-space tool to read existing SQL databases, so adding the user-space tool to the firmware may have limited benefit, given most environments install Entware anyway.

..but +1 anyway.

But I suspect more would use it if it was part of the firmware instead of requiring Entware.

I went ahead and added it since it was such a small binary.

 

[ColinTaylor]

Just the other day I was toying with the idea of using sqlite to store Diversions config settings and more.

Just curious (because I don't use Diversion) about how many different config settings you have? Generally speaking I'm strongly opposed to anything that stores simple data in a non-human-readable, binary format that relies on a specific tool to edit it "just because you can". OTOH I suppose it lets you easily update individual settings without having to re-write the entire file.

 

[thelonelycoder]

Just curious (because I don't use Diversion) about how many different config settings you have? Generally speaking I'm strongly opposed to anything that stores simple data in a non-human-readable, binary format that relies on a specific tool to edit it "just because you can". OTOH I suppose it lets you easily update individual settings without having to re-write the entire file.

Depending on the selected options, up to about 80. The problem I have is the locking and simultaneous writing to it. I try as best as I can to prevent it but the possibility is there that parts are not updated correctly when several processes want to write to the config file.
The main concern is the ads counters numbers, really.

I am looking at a way to have a lock state just as MySQL or MariaDB have.

 

[Bamsefar]

Will this replace Entware, then I am for this. However if all tools under AMTM still requires Entware I see no use for this. In my mind it is more a question like either Entware or sqlite3...?

 

[L&LD]

Will this replace Entware, then I am for this. However if all tools under AMTM still requires Entware I see no use for this. In my mind it is more a question like either Entware or sqlite3...?

No, it can't replace Entware, it is a program Entware installs.

 

[Bamsefar]

No, it can't replace Entware, it is a program Entware installs.

Exactly my point ;-)

 

[L&LD]

Exactly my point ;-)

Then you may be missing the point? Or, I am.

 

[kernol]

@RMerlin - time to close this thread ... confusion begins to reign and you have already done what you asked whether you should

To conclude - sqlite3 user tool has now been included in MerlinWare and will be available in future releases;
Sqlite3 is independent of Entware - so will be available for coder use whether they also need Entware or even if they don't.