waiverdb

#124 Split migration.
Merged by ralph. Opened by ralph.
splitting-migraine  into  master

Download 124.patch

For #121 and #115.

Interesting. At least partial success here.

λ oc logs waiverdb-test-jenkins-waiverdb-rbean-1-web-1-deploy                 ~
--> pre: Running hook pod ...
sh: cannot set terminal process group (-1): Inappropriate ioctl for device
sh: no job control in this shell
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 0a27a8ad723a, Initializes migration
INFO  [alembic.runtime.migration] Running upgrade 0a27a8ad723a -> 0a74cdab732a, add proxyuser waiving support
INFO  [alembic.runtime.migration] Running upgrade 0a74cdab732a -> f2772c2c64a6, waive absence of result
INFO  [alembic.runtime.migration] Running upgrade f2772c2c64a6 -> 71b84ccc31bb, migrate records from old format to new.
INFO  [alembic.runtime.migration] Running upgrade 71b84ccc31bb -> ed43eb9b221c, set nullable on new and old fields.

It does seem to hang on that last upgrade though...

(waiverdb) waiverdb λ oc status                                                                                                                                                  devel/waiverdb splitting-migraine
In project WaiverDB test environments (waiverdb-test) on server https://open.paas.redhat.com:443
svc/waiverdb-test-jenkins-waiverdb-rbean-1-database - 172.30.242.203:5432
  dc/waiverdb-test-jenkins-waiverdb-rbean-1-database deploys registry.access.redhat.com/rhscl/postgresql-95-rhel7:latest 
    deployment #1 deployed 5 minutes ago - 1 pod
https://waiverdb-test-jenkins-waiverdb-rbean-1-web-waiverdb-test.int.open.paas.redhat.com (redirects) to pod port web (svc/waiverdb-test-jenkins-waiverdb-rbean-1-web)
  dc/waiverdb-test-jenkins-waiverdb-rbean-1-web deploys docker-registry.engineering.redhat.com/factory2/waiverdb:0.6.1.dev9-git.49e6947 
    deployment #1 running for 5 minutes
View details with 'oc describe <resource>/<name>' or list everything with 'oc get all'.
(waiverdb) waiverdb λ oc get pods                                                                                                                                                devel/waiverdb splitting-migraine
NAME                                                      READY     STATUS    RESTARTS   AGE
waiverdb-test-jenkins-waiverdb-rbean-1-database-1-dl8vv   1/1       Running   0          5m
waiverdb-test-jenkins-waiverdb-rbean-1-web-1-deploy       1/1       Running   0          5m
waiverdb-test-jenkins-waiverdb-rbean-1-web-1-hook-pre     1/1       Running   0          5m

Yeah, it's hung:

waiverdb=> SELECT * FROM pg_stat_activity WHERE waiting;
 datid | datname  | pid | usesysid | usename  | application_name | client_addr | client_hostname | client_port |        backend_start         |          xact_start           |         query_start          |         state_change         | waiting | state  | backend_xid | backend_xmin |                        query                         
-------+----------+-----+----------+----------+------------------+-------------+-----------------+-------------+------------------------------+-------------------------------+------------------------------+------------------------------+---------+--------+-------------+--------------+------------------------------------------------------
 16385 | waiverdb | 121 |    16384 | waiverdb |                  | 10.1.24.1   |                 |       51756 | 2018-02-14 17:41:59.84921+00 | 2018-02-14 17:41:59.906088+00 | 2018-02-14 17:41:59.90625+00 | 2018-02-14 17:41:59.90625+00 | t       | active |             |          629 | ALTER TABLE waiver ALTER COLUMN subject SET NOT NULL
(1 row)

1 new commit added

  • Try dropping batch operation to get postgres upgrades working.

2 new commits added

  • Try dropping batch operation to get postgres upgrades working.
  • Split migration.

2 new commits added

  • Some log statements.
  • Split migration.

1 new commit added

  • Put this operation on a session associated with the alembic op.

3 new commits added

  • Put this operation on a session associated with the alembic op.
  • Some log statements.
  • Split migration.

3 new commits added

  • Put this operation on a session associated with the alembic op.
  • Some log statements.
  • Split migration.

Ah, that finally did it. See run number 9 of the waiverdb-rbean jenkins job on the rcm-tools-jenkins instance. :)

Output from the pre hook:

λ oc logs -f waiverdb-test-jenkins-waiverdb-rbean-9-web-1-hook-pre   devel/waiverdb splitting-migraine 
sh: cannot set terminal process group (-1): Inappropriate ioctl for device                             
sh: no job control in this shell                   
INFO  [alembic.env] Connecting...                  
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.                                         
INFO  [alembic.runtime.migration] Will assume transactional DDL.                                       
INFO  [alembic.env] Beginning outermost transaction.                                                   
INFO  [alembic.env] Starting migrations.           
INFO  [alembic.runtime.migration] Running upgrade  -> 0a27a8ad723a, Initializes migration              
INFO  [alembic.runtime.migration] Running upgrade 0a27a8ad723a -> 0a74cdab732a, add proxyuser waiving support
INFO  [alembic.runtime.migration] Running upgrade 0a74cdab732a -> f2772c2c64a6, waive absence of result
INFO  [alembic.runtime.migration] Running upgrade f2772c2c64a6 -> 71b84ccc31bb, migrate records from old format to new.
INFO  [alembic.runtime.migration] Running upgrade 71b84ccc31bb -> ed43eb9b221c, set nullable on new and old fields.
INFO  [alembic.env] Done with migrations.          
INFO  [alembic.env] Outermost transaction released.                                                    
INFO  [alembic.env] Closing connection.

I think this could just be:

db.session.bind = op.get_bind()

and then use the normal db stuff below. But this is okay too.

Nice :+1:

Pull-Request has been merged by ralph
Metadata
None
No Tags
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.