Re: [libreoffice-users] mongodb and base

On Nov 22, 2014 5:53 PM, Paul <paulsteyn1@afrihost.co.za> wrote:
> I don't want to try and tell you your business, but I had a few 
> thoughts on your mail:

No problems. You have some very good feedback which I appreciate and I couldn't say much because I 
only had the roughest of conversations with this client. It's still not fully fleshed out but, 
there is more political issues than technical.

> On Mon, 17 Nov 2014 09:18:43 -0500 
> Eric <esj@esjworks.com> wrote: 
>
> > On 11/16/2014 5:16 PM, Jaroslaw Staniek wrote: 
> > > Do you have use cases? 
> > I'm preping to talk to a real estate title authority.  they have 
> > 60+years of title info in the area and it is all held in informix. 
>
> That's a lot of data. And Informix is a real workhorse of a database. 
> Arbitrarily shifting to MongoDB and Base just doesn't sound like a well 
> though out decision...

It was more exploratory thought. Informix is an extremely expensive database, IBM stripped out 
essential functionality and the current database is running on RedHat 5 and poorly maintained. 
Informix is similarly old and they will not upgrade. Its been reported that the cost of the 
Informix database plus any number of third party report writers would cost them 10 years of revenue.

I have a lot more conversations to go before I can validate their financial assessments of the 
conversion.

> Base doesn't strike me as a rock solid, "serious" solution, but that 
> might be just my perception.

Probably is it. I was trying to explore options for two reasons. I should have been clear about 
separating out the two.

However, there are a few more pointers in 
> your mail that suggest the decision may be less than considered: 
>
>
> > they generate reports on the fly (hence base) and would like to be 
>
> What exactly do you mean by this? That they regularly want custom 
> designed reports? Sounds odd. Surely they must have some idea of what 
> they want to get out of the system on a regular basis? Otherwise it 
> implies that they don't really know what they want the system for. And 
> that right there would be a fundamental problem, one that no amount of 
> engineering will solve.

That solution is unfortunately locked up in political challenges that will not be resolved until 1 
of the lawyers retires. But from what I've seen, there is a significant need for flexibility in 
report generation. How much is a real can't be revealed until they let me understand more about how 
they do their work.
> > able to add information on properties incrementally without always 
> > rewriting the schema and go through the database conversion. 
>
> A common request, but almost always a bad idea. By "adding information" 
> you must mean adding fields to the database, which should always be a 
> carefully considered decision, not an "as the mood strikes" decision.

Given the nature the information and their standing as a title authority, I can assure you that the 
rate of change for record editions is very slow and considered.
> Assuming that such fields are purely additional data, and in no way 
> form part of any relationships, then sure, they can be added quite 
> easily, but the question is always who is going to go through all the 
> current entries in the database filling in that new field? One can make 
> the new fields nullable, and just let all old data have nulls for those 
> new fields, but that just leaves a mess of incomplete data in the 
> database. If they've been collecting data for the past 60+ years, they 
> must have a pretty good idea of what data they need. Real estate data 
> can't change that often.

From what I can gather, you're mostly right. The primary change is our building together more 
detailed information about properties that we had in the past and do it more accurately. I think 
they're also changing legal requirements on titles with regards to title searches and their 
validity. Fortunately, those changes are very very slow.

> > Yes, there may some legal reasons why they can't do this but htat is 
> > why er are talking. 
>
> Legal reasons? I can't think of any. This is their data, they can do 
> what they like with it, I should think. But I'm a technical guy, not a 
> legal guy, so I really have little idea. All I *can* tell you is that 
> from a technical standpoint this sounds like a nightmare waiting to 
> bite you in the posterior later. And the fact that you think there may 
> be legal issues may indicate a misunderstanding about what it is the
> client is trying to do.

When it comes to this kind of information, there are all sorts of weird legal quirks built up as a 
result of hundreds of years of additive legal precedent. I know enough about their world there to 
consider walking away from this contract because of the internal bickering.  , I probably just 
described the best reason for walking away. :-)
> > personally, I am an order of magnitude more effective with mongo than 
> > any sql database.  mongo maps better to my mind than sql.  I'd like a 
> > base connection so I can be more effective in my ad-hoc db tools 
>
> At a fundamental level, it sounds like the problem is with the concept, 
> not the implementation. I'd go back to the drawing board on this one. 
> Rethink what your client wants to do, and from there use proper tools 
> to implement it. Much as I am a fan of LO, I just can't recommend Base 
> as a tool for serious work in the order of 60+ years of data in an 
> Informix database.

I'm in the process of figuring out what they really want instead of what they tell me what to do. 
But when it comes down to it, the most common complaint I hear is that it just cost too damn much 
and they feel a bit ripped off by IBM.

Between license fees and the lost report writing capability that can not be replaced with third 
party, they are feeling kind of burned and between a rock and a hard place.
> But then again, you know what the client wants, I don't. And as you're 
> the one who has to do the work, you will need to decide how closely you 
> want to stick with what you're familiar with. And one assumes you're 
> the guy who has to live with the decisions. So if you want MongoDB and 
> Base, I'm sure someone will be able to point you in the right 
> direction. Unfortunately I can't, as I have no experience with MongoDB.

Remember, using MongoDB and base are two separate decisions. I was trying to close out the matrix 
and see if I could use beast to replace the report and data entry tools now lost to Informix.
> I just thought I'd point out what seems like a mess. You might well 
> have thought it through thoroughly, and this advice may be unwarranted 
> and unwanted, and if so I apologise, but I felt it better to offer 
> caution than to let you step blindly into harm's way. 
>
> It's entirely your call of course, but, given that I've done these 
> sorts of systems for a living for quite a few years, your description 
> raised warnings when parsed through my experience filter, and I felt I 
> should tell you about it.

No problem, I appreciate the input. Unfortunately, my experience with SQL over the past 30 years 
has taught me to stay away from SQL as far as absolutely possible. I know most of my problems with 
SQL come from an impedance mismatch between how I solve problems and how it wants to express 
solutions. Usually, the group of people I work with leave the database to the very last and 
isolated. We do that so the rest of the application is not contaminated by SQL isms and we have a 
nailed down definition of the schema. We don't have hard numbers yet but with MongoDB, we are not 
getting any of the chaos and heartache that SQL delivers on a regular basis.

I really do wish I SQL would go the way of COBOL.

-- 
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted