# 3 machines
# obriensystems dev laptop
# gerrit source server
# gerrit2 replication server
# on remote dev host - against gerrit
git clone "ssh://admin@gerrit.ons.zone:29418/test" && scp -p -P 29418 admin@gerrit.ons.zone:hooks/commit-msg "test/.git/hooks/"
cd test/
vi test.sh
git add test.sh
git commit -s --amend
git review
# getting merge conflict - needed to remove old commit id
vi test.sh
git add test.sh
git rebase --continue
git review
# move to gerrit UI, +2 review, merge
# on gerrit server
ssh ubuntu@gerrit.ons.zone
# tail the logs to the gerrit container
# on dev laptop
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone gerrit plugin reload replication
[2019-03-28 15:25:57,246] [SSH gerrit plugin reload replication (admin)] INFO com.google.gerrit.server.plugins.PluginLoader : Reloaded plugin replication, version v2.16.6
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone replication list
Remote: gerrit2
Url: admin@gerrit2.ons.zone:8080/${name}.git
[2019-03-28 15:26:57,963] [WorkQueue-1] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_replication_190328_0446_6094540689096397413.jar
# debug on
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set DEBUG
# debug off
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set reset
ssh -p 29418 admin@gerrit.ons.zone replication start --wait --all
# nothing yet - debugging container I only see a recent
var/gerrit/data/replication/ref-updates
-rw-r--r-- 1 gerrit gerrit 45 Mar 28 15:25 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066
bash-4.2$ cat 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066
{"project":"test","ref":"refs/heads/master"}
Issue was the key - after changing the url to
url = admin@gerrit2.ons.zone:29418/${name}.git
I can ssh directly from gerrit to gerrit2 but they key is n/a for the container yet
sshd_log
[2019-03-28 15:57:50,164 +0000] b2bd0870 admin a/1000000 replication.start.--all 3ms 1ms 0
replication_log
[2019-03-28 17:34:07,816] [72da30d3] Replication to admin@gerrit2.ons.zone:29418/All-Users.git started...
[2019-03-28 17:34:07,834] [72da30d3] Cannot replicate to admin@gerrit2.ons.zone:29418/All-Users.git
org.eclipse.jgit.errors.TransportException: admin@gerrit2.ons.zone:29418/All-Users.git: reject HostKey: gerrit2.ons.zone
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:192)
# I am running hashed
ubuntu@ip-172-31-15-176:~$ grep "HashKnownHosts" /etc/ssh/ssh_config
HashKnownHosts yes
# tried - it may be my url
Url: ssh://admin@gerrit2.ons.zone:29418/${name}.git
from
Url: admin@gerrit2.ons.zone:29418/${name}.git
[2019-03-28 21:54:04,089] [] Canceled 3 replication events during shutdown
[2019-03-28 21:54:17,738] [] scheduling replication All-Projects:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
[2019-03-28 21:54:17,750] [] scheduled All-Projects:..all.. => [283d568e] push ssh://admin@gerrit2.ons.zone:29418/All-Projects.git to run after 15s
[2019-03-28 21:54:17,750] [] scheduling replication All-Users:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Users.git
[2019-03-28 21:54:17,751] [] scheduled All-Users:..all.. => [684a6e1d] push ssh://admin@gerrit2.ons.zone:29418/All-Users.git to run after 15s
[2019-03-28 21:54:17,751] [] scheduling replication test:..all.. => ssh://admin@gerrit2.ons.zone:29418/test.git
[2019-03-28 21:54:17,751] [] scheduled test:..all.. => [a84066fe] push ssh://admin@gerrit2.ons.zone:29418/test.git to run after 15s
[2019-03-28 21:54:32,751] [283d568e] Replication to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git started...
[2019-03-28 21:54:32,857] [283d568e] Cannot replicate to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
org.eclipse.jgit.errors.TransportException: ssh://admin@gerrit2.ons.zone:29418/All-Projects.git: reject HostKey: gerrit2.ons.zone
Hostname is generated by docker - overriding
using hostname not hostname.domainname pair - as gerrit will only pick up the prefix
services:
gerrit:
image: gerritcodereview/gerrit
hostname: gerrit2.ons.zone
ubuntu@ip-172-31-6-115:~$ sudo docker exec -it ubuntu_gerrit_1 bash
bash-4.2$ hostname
gerrit2.ons.zone
#trying protocol 1 (insecure) instead of 2
/home/ubuntu/.ssh/config line 5: Bad protocol spec '1'.
Verifying via https://gerrit.googlesource.com/plugins/replication/+/master/src/main/resources/Documentation/config.md
PreferredAuthentications publickey
Working with Igor - trying http instead of ssh
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
2019-03-29 19:54:23,740] [6737289e] Cannot replicate replicate; Remote repository error: server
# gerrit2 replication server
# on remote dev host - against gerrit
git clone "ssh://admin@gerrit.ons.zone:29418/test" && scp -p -P 29418 admin@gerrit.ons.zone:hooks/commit-msg "test/.git/hooks/"
cd test/
vi test.sh
git add test.sh
git commit -s --amend
git review
# getting merge conflict - needed to remove old commit id
vi test.sh
git add test.sh
git rebase --continue
git review
# move to gerrit UI, +2 review, merge
# on gerrit server
ssh ubuntu@gerrit.ons.zone
# tail the logs to the gerrit container
# on dev laptop
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone gerrit plugin reload replication
[2019-03-28 15:25:57,246] [SSH gerrit plugin reload replication (admin)] INFO com.google.gerrit.server.plugins.PluginLoader : Reloaded plugin replication, version v2.16.6
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone replication list
Remote: gerrit2
Url: admin@gerrit2.ons.zone:8080/${name}.git
[2019-03-28 15:26:57,963] [WorkQueue-1] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_replication_190328_0446_6094540689096397413.jar
# debug on
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set DEBUG
# debug off
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set reset
ssh -p 29418 admin@gerrit.ons.zone replication start --wait --all
# nothing yet - debugging container I only see a recent
var/gerrit/data/replication/ref-updates
-rw-r--r-- 1 gerrit gerrit 45 Mar 28 15:25 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066
bash-4.2$ cat 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066
{"project":"test","ref":"refs/heads/master"}
Issue was the key - after changing the url to
url = admin@gerrit2.ons.zone:29418/${name}.git
I can ssh directly from gerrit to gerrit2 but they key is n/a for the container yet
sshd_log
[2019-03-28 15:57:50,164 +0000] b2bd0870 admin a/1000000 replication.start.--all 3ms 1ms 0
replication_log
[2019-03-28 17:34:07,816] [72da30d3] Replication to admin@gerrit2.ons.zone:29418/All-Users.git started...
[2019-03-28 17:34:07,834] [72da30d3] Cannot replicate to admin@gerrit2.ons.zone:29418/All-Users.git
org.eclipse.jgit.errors.TransportException: admin@gerrit2.ons.zone:29418/All-Users.git: reject HostKey: gerrit2.ons.zone
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:192)
# I am running hashed
ubuntu@ip-172-31-15-176:~$ grep "HashKnownHosts" /etc/ssh/ssh_config
HashKnownHosts yes
# tried - it may be my url
Url: ssh://admin@gerrit2.ons.zone:29418/${name}.git
from
Url: admin@gerrit2.ons.zone:29418/${name}.git
[2019-03-28 21:54:04,089] [] Canceled 3 replication events during shutdown
[2019-03-28 21:54:17,738] [] scheduling replication All-Projects:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
[2019-03-28 21:54:17,750] [] scheduled All-Projects:..all.. => [283d568e] push ssh://admin@gerrit2.ons.zone:29418/All-Projects.git to run after 15s
[2019-03-28 21:54:17,750] [] scheduling replication All-Users:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Users.git
[2019-03-28 21:54:17,751] [] scheduled All-Users:..all.. => [684a6e1d] push ssh://admin@gerrit2.ons.zone:29418/All-Users.git to run after 15s
[2019-03-28 21:54:17,751] [] scheduling replication test:..all.. => ssh://admin@gerrit2.ons.zone:29418/test.git
[2019-03-28 21:54:17,751] [] scheduled test:..all.. => [a84066fe] push ssh://admin@gerrit2.ons.zone:29418/test.git to run after 15s
[2019-03-28 21:54:32,751] [283d568e] Replication to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git started...
[2019-03-28 21:54:32,857] [283d568e] Cannot replicate to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
org.eclipse.jgit.errors.TransportException: ssh://admin@gerrit2.ons.zone:29418/All-Projects.git: reject HostKey: gerrit2.ons.zone
Hostname is generated by docker - overriding
using hostname not hostname.domainname pair - as gerrit will only pick up the prefix
services:
gerrit:
image: gerritcodereview/gerrit
hostname: gerrit2.ons.zone
ubuntu@ip-172-31-6-115:~$ sudo docker exec -it ubuntu_gerrit_1 bash
bash-4.2$ hostname
gerrit2.ons.zone
#trying protocol 1 (insecure) instead of 2
/home/ubuntu/.ssh/config line 5: Bad protocol spec '1'.
Verifying via https://gerrit.googlesource.com/plugins/replication/+/master/src/main/resources/Documentation/config.md
PreferredAuthentications publickey
Working with Igor - trying http instead of ssh
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
2019-03-29 19:54:23,740] [6737289e] Cannot replicate replicate; Remote repository error: http://admin@gerrit2.ons.zone:8080/replicate.git: replicate unavailable
[2019-03-29 19:54:23,741] [a71c401e] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 19:54:23,754] [e7c79889] Replication to http://admin@gerrit2.ons.zone:8080/All-Projects.git completed in 121ms, 15001ms delay, 0 retries
[2019-03-29 19:54:23,756] [273130a7] Replication to http://admin@gerrit2.ons.zone:8080/All-Users.git completed in 116ms, 15000ms delay, 0 retries
[2019-03-29 19:54:23,762] [a71c401e] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...e9efbebcb130387cfa65e6f3c47dea5d005f8bbe, srcRef=refs/heads/master, forceUpdate, message=null]]
[2019-03-29 19:54:24,093] [a71c401e] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/test.git, reason: prohibited by Gerrit: not permitted: force update
[2019-03-29 19:54:24,094] [a71c401e] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 352ms, 15103ms delay, 0 retries
changed signed to true
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
[2019-03-29 20:11:48,466] [a722008c] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 20:11:48,467] [a722008c] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...a9f165b560889e937a10ac45f425c6d727a8fb78, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/changes/03/1003/1, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/changes/03/1003/1, forceUpdate, message=null]]
[2019-03-29 20:11:48,662] [a722008c] Failed replicate of refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes
[2019-03-29 20:11:48,662] [a722008c] Failed replicate of refs/changes/03/1003/1 to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes
[2019-03-29 20:11:48,663] [a722008c] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 197ms, 15000ms delay, 0 retries
review+2/committed
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
[2019-03-29 20:15:12,674] [a7d4407f] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 20:15:12,676] [a7d4407f] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...60d083b4f4b0916a6dc0da694c52a5e7ff08a9b7, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/heads/master, forceUpdate, message=null]]
[2019-03-29 20:15:12,862] [a7d4407f] Failed replicate of refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git, reason: NoteDb update requires -o notedb=allow
[2019-03-29 20:15:12,862] [a7d4407f] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/test.git, reason: prohibited by Gerrit: not permitted: force update
[2019-03-29 20:15:12,863] [a7d4407f] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 188ms, 15000ms delay, 0 retries
gerrit.defaultForceUpdate : If true, the default push refspec will be set to use forced update to the remote when no refspec is given. By default, false.
[gerrit]
defaultForceUpdate = true
still magic error - added above to target gerrit
https://groups.google.com/forum/#!msg/repo-discuss/m2E72F2oiuo/w-lWg0WUZYIJ
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
[[A[2019-03-29 21:03:24,045] [] scheduling replication All-Projects:..all.. => http://admin@gerrit2.ons.zone:8080/All-Projects.git
[2019-03-29 21:03:24,068] [] scheduled All-Projects:..all.. => [71c8678f] push http://admin@gerrit2.ons.zone:8080/All-Projects.git to run after 15s
[2019-03-29 21:03:24,071] [] scheduling replication All-Users:..all.. => http://admin@gerrit2.ons.zone:8080/All-Users.git
[2019-03-29 21:03:24,072] [] scheduled All-Users:..all.. => [b1cedf9b] push http://admin@gerrit2.ons.zone:8080/All-Users.git to run after 15s
[2019-03-29 21:03:24,075] [] scheduling replication replicate:..all.. => http://admin@gerrit2.ons.zone:8080/replicate.git
[2019-03-29 21:03:24,076] [] scheduled replicate:..all.. => [f19bf7a7] push http://admin@gerrit2.ons.zone:8080/replicate.git: replicate unavailable to run after 15s
[2019-03-29 1921:5403:2324,741079] [a71c401e] Replication to scheduling replication test:..all.. => http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 19:54:23,754] [e7c79889] Replication to21:03:24,080] [] scheduled test:..all.. => [3192ef91] push http://admin@gerrit2.ons.zone:8080/All-Projectstest.git completedto inrun 121ms, 15001ms delay, 0 retries
after 15s
[2019-03-29 1921:5403:2339,756070] [273130a771c8678f] Replication to http://admin@gerrit2.ons.zone:8080/All-UsersProjects.git completed in 116ms, 15000ms delay, 0 retriesstarted...
[2019-03-29 1921:5403:2339,762072] [a71c401eb1cedf9b] PushReplication to http://admin@gerrit2.ons.zone:8080/testAll-Users.git references: [RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...e9efbebcb130387cfa65e6f3c47dea5d005f8bbe, srcRef=refs/heads/master, forceUpdate, message=null]]started...
[2019-03-29 1921:5403:2439,093076] [a71c401ef19bf7a7] Failed replicate of refs/heads/master Replication to http://admin@gerrit2.ons.zone:8080/testreplicate.git, reason: prohibited by Gerrit: not permitted: force update
started...
[2019-03-29 1921:5403:2439,094245] [a71c401e71c8678f] Replication to http://admin@gerrit2.ons.zone:8080/testAll-Projects.git completed in 352ms173ms, 15103ms15004ms delay, 0 retries
[2019-03-29 21:03:39,246]
changed signed[3192ef91] Replication to true
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 21:03:39,245] [f19bf7a7] Push to http://admin@gerrit2.ons.zone:8080/replicate.git references: [RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...410858cdd130ee2d56700d199f4021c246c1d22b, srcRef=refs/heads/master, forceUpdate, message=null]]
[2019-03-29 2021:1103:4839,466245] [a722008cb1cedf9b] Replication to http://admin@gerrit2.ons.zone:8080/testAll-Users.git started... completed in 173ms, 15000ms delay, 0 retries
[2019-03-29 2021:1103:4839,467281] [a722008c3192ef91] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta1, NOT_ATTEMPTED, (null)...a9f165b560889e937a10ac45f425c6d727a8fb78477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/changes/03/1003/meta1, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/changes/03/1003/1meta, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c260d083b4f4b0916a6dc0da694c52a5e7ff08a9b7, srcRef=refs/changes/03/1003/1meta, forceUpdate, message=null]]
[2019-03-29 20:11:48,662] [a722008c] Failed replicate of RemoteRefUpdate[remoteName=refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes
[2019-03-29 20:11:48,662] [a722008c] Failed replicate of refs/changes/03/1003/1 to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushesheads/master, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/heads/master, forceUpdate, message=null]]
[2019-03-29 2021:1103:4839,663682] [a722008c3192ef91] Failed Replicationreplicate toof http:refs//admin@gerrit2.ons.zone:8080/test.git completed in 197ms, 15000ms delay, 0 retries
review+2/committed
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}
changes/03/1003/1 to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes
[2019-03-29 2021:1503:1239,674682] [a7d4407f3192ef91] ReplicationFailed replicate of refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git started..., reason: cannot combine normal pushes and magic pushes
[2019-03-29 2021:1503:1239,676682] [a7d4407f] Push3192ef91] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/test.git, referencesreason: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...60d083b4f4b0916a6dc0da694c52a5e7ff08a9b7, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/heads/master, forceUpdate, message=null]]
cannot combine normal pushes and magic pushes
[2019-03-29 2021:1503:1239,862682] [a7d4407f3192ef91] Failed replicate of refs/changes/03/1003/metaReplication to http://admin@gerrit2.ons.zone:8080/test.git, reason: NoteDb update requires -o notedb=allow completed in 435ms, 15166ms delay, 0 retries
[2019-03-29 2021:1503:1239,862704] [a7d4407ff19bf7a7] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/testreplicate.git, reason: prohibited by Gerrit: not permitted: force update
[2019-03-29 2021:1503:1239,863705] [a7d4407ff19bf7a7] Replication to http://admin@gerrit2.ons.zone:8080/testreplicate.git completed in 188ms628ms, 15000ms delay, 0 retries
gerrit.defaultForceUpdate : If true, the default push refspec will be set to use forced update to the remote when no refspec is given. By default, false.
[gerrit]
defaultForceUpdate = true
|