Ticket #176 (closed enhancement: fixed)

Opened 5 years ago

Last modified 3 years ago

South migrate --status (shows differences, details of migration runs)

Reported by: Robert <trebor74hr@…> Owned by: andrew
Priority: major Milestone: 0.7
Component: commands Version: 0.6-pre
Keywords: Cc:

Description (last modified by andrew) (diff)

Proposal for new command:

manage migrate --status <app> <migration>

This can list following information about app.migration:

  • compare db with migration scheme (frozen)- if diff - then show differences
  • show south_migrationhistory details (time, status, etc.)
  • show logs of last migration try

For this we should need to have following new features:

  • compare db scheme with south migration scheme (frozen one) - use django introspection - if there is difference - then show differences. This can become handy when checking the integrity of the system and/or manually fixing failures (see also CONFL in http://south.aeracode.org/ticket/173#comment:2).

Change History

comment:1 Changed 5 years ago by Robert <trebor74hr@…>

I suggest south_log model to have following fields:

  • fk to south_migrationhistory
  • order number - for the same south_migrationhistory
  • timestamp when entry is created
  • modelname (can be empty)
  • method_name (e.g.add_column, delete_table)
  • columnname (can be empty)
  • detailed description
  • status - choices (e.g. OK, ROLL, CONFL like proposed in #173)

Model._meta.ordering = ["migration", "order_nr"]. In order to preserve all that happened to database, south_migrationhistory entries should not be deleted (or even updated?) after creation.

comment:2 Changed 5 years ago by andrew

  • Status changed from new to assigned
  • Version set to subversion
  • Milestone set to 0.6

Interesting. This is, essentially, two separate proposals, and I shall address it as such:

  • Firstly, the general idea of a --status. This goes above --list by adding information about currently-detected changes (useful), when things were done and how it went (useful), and 'logs' (see below).
  • The second part is this logging. Now, while I love the idea of logging in principle, there's a few problems with this idea and the proposed schema.

Firstly, migrations don't particularly affect a certain model or use a certain method or touch a certain column; they're allowed to do anything, which includes fiddling with data and using raw SQL, so this level of logging isn't going to be possible.

I think just having things like in #173 is better - the only thing South can reliably determine is which migration ran and how it did. It's possible I can add in a logging hook so those so inclined can log a blow-by-blow history of what was applied when (including rollbacks), for e.g. production servers, but I'd do this to a file rather than a table (for various reasons, including logging failures due to databases vanishing).

If you don't object, I'll change this to a ticket about --status only, and perhaps a textual logging - I personally don't see a need for this not-always-achievable, fine-grained logging.

comment:3 Changed 5 years ago by Robert <trebor74hr@…>

I would suggest to split two issues in two tickets - this ticket for status and logging should go to the new ticket?

Having logs in textual files is better in the way of stability. But if you can't write to db, you won't be able to do migration and/or mark it as OK/failed too (south_migrationhistory). But there is an issue of rollback, so text logging is maybe the better solution.

Anyway, if logging should be in textual file, my suggestion is to be in structual fmt, easy for parsing (e.g. fixed columns, separate with tab or similar). Also location/names of the log files to be predefined (e.g. migrations/logs/<migration_id>_<datetime>.log)

I think it is very comfortable to see everything that system tried on existing database in the past (with info what was ok, what went wrong - and err message), and connected to correct south_migrationhistory record - (head record). For some bigger migration sometimes timing is also important information.

I didn't understand fully the problem about logging hooks you wrote about. Is this possible:
Since every south migration operation goes through south.db.generic.DatabaseOperations? method, putting at beginning/end of each method self.log(...) could solve most of the log hooks everything. For some db specific methods which don't call parent method we could rename such method to be _<method_name> in generic and db-specific and create new DatabaseOperations?.<method_name> which calls _<method_name> internally (proxying).

    generic.def create_table(...):
    ->
    generic.def _create_table(...):

    db_specific.def create_table(...):
    ->
    db_specific.def _create_table(...):

create new
    generic.def _create_table(...):
        self.log("started...")
        self._create_table(...)
        self.log("ended...")

Note that migration.log is public method and user can call it explicitely in migration too.

comment:4 Changed 5 years ago by Robert Lujo <trebor74hr@…>

Typo:

create new
    generic.def create_table(...):

comment:5 Changed 5 years ago by andrew

  • Description modified (diff)
  • Summary changed from South migrate --status, compare db with south scheme, logging south actions to db to South migrate --status (shows differences, details of migration runs)

Yes, I agree, this ticket should become status only (I've marked it as so).

As for logging issues, I was under the incorrect assumption you wanted one log entry per migration - sorry! There are still issues, however, with people using the ORM, but I can probably at least log which methods they call (since the ORM's managers are wrapped by South).

comment:6 Changed 5 years ago by andrew

Logging is now in #178.

comment:7 Changed 5 years ago by andrew

  • Milestone changed from 0.6 to 0.7

Bumping to 0.7 - some things require #173, and so might get bumped, and some will be done. --status's show-changes feature can somewhat be replicated using --auto --stdout.

comment:8 Changed 5 years ago by andrew

  • Status changed from assigned to closed
  • Resolution set to fixed

Deciding not to implement this, it's easily replicated with:

./manage.py schemamigration appname --auto - > /dev/null
Note: See TracTickets for help on using tickets.