Versions Compared

Old Version 46

changes.mady.by.user Michael O'Brien

Saved on

compared with

New Version Current

changes.mady.by.user Michael O'Brien

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Gerrit

Config Jobs

login as default admin

create test repo

Gerrit repo url


PostgreSQL

Default Helm Chart

https://github.com/helm/charts/tree/master/stable/postgresql

# 3 machines # obriensystems dev laptop # gerrit source server # gerrit2 replication server # on remote dev host - against gerrit git clone "ssh://admin@gerrit.ons.zone:29418/test" && scp -p -P 29418 admin@gerrit.ons.zone:hooks/commit-msg "test/.git/hooks/" cd test/ vi test.sh git add test.sh git commit -s --amend git review # getting merge conflict - needed to remove old commit id vi test.sh git add test.sh git rebase --continue git review # move to gerrit UI, +2 review, merge # on gerrit server ssh ubuntu@gerrit.ons.zone # tail the logs to the gerrit container # on dev laptop obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone gerrit plugin reload replication [2019-03-28 15:25:57,246] [SSH gerrit plugin reload replication (admin)] INFO com.google.gerrit.server.plugins.PluginLoader : Reloaded plugin replication, version v2.16.6 obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone replication list Remote: gerrit2 Url: admin@gerrit2.ons.zone:8080/${name}.git [2019-03-28 15:26:57,963] [WorkQueue-1] INFO com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_replication_190328_0446_6094540689096397413.jar # debug on ssh -p 29418 admin@gerrit.ons.zone gerrit logging set DEBUG # debug off ssh -p 29418 admin@gerrit.ons.zone gerrit logging set reset ssh -p 29418 admin@gerrit.ons.zone replication start --wait --all # nothing yet - debugging container I only see a recent var/gerrit/data/replication/ref-updates -rw-r--r-- 1 gerrit gerrit 45 Mar 28 15:25 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066 bash-4.2$ cat 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066 {"project":"test","ref":"refs/heads/master"} Issue was the key - after changing the url to url = admin@gerrit2.ons.zone:29418/${name}.git I can ssh directly from gerrit to gerrit2 but they key is n/a for the container yet sshd_log [2019-03-28 15:57:50,164 +0000] b2bd0870 admin a/1000000 replication.start.--all 3ms 1ms 0 replication_log [2019-03-28 17:34:07,816] [72da30d3] Replication to admin@gerrit2.ons.zone:29418
Code Block
Code Block
themeRDark
themeRDark
verified
git clone ssh://admin@gerrit2.ons.zone:29418/test.git
remote: Counting objects: 2, done
git clone admin@gerrit2.ons.zone:29418/test.git
admin@gerrit2.ons.zone: Permission denied (publickey).


verified on gerrit3 to 2 host
ubuntu@ip-172-31-31-191:~$ git clone  ssh://admin@gerrit2.ons.zone:29418/replicate.git
Cloning into 'replicate'...


verified admin user
ubuntu@ip-172-31-31-191:~$ ssh -p 29418 admin@gerrit2.ons.zone
Warning: Permanently added '[gerrit2.ons.zone]:29418,[3.17.20.86]:29418' (RSA) to the list of known hosts.
  Hi Administrator, you have successfully connected over SSH.

gerrit source - /var/gerrit/etc/gerrit.config

Code Block
themeRDark
# adjust server name
[gerrit]
        #canonicalWebUrl = http://fcdbe931c71d
        canonicalWebUrl = http://gerrit2.ons.zone

[receive]
        #enableSignedPush = false
        enableSignedPush = truef

gerrit source - /var/gerrit/etc/replication.config

Code Block
themeRDark
[remote "gerrit2"]
  url = ssh://admin@gerrit2.ons.zone:29418/${name}.git
  push = +refs/heads/*:refs/heads/*
  push = +refs/tags/*:refs/tags/*

Host ~/.ssh/config

Code Block
themeMidnight
Host remote-alias gerrit2.ons.zone
  IdentityFile ~/.ssh/onap_rsa
  Hostname gerrit2.ons.zone
  Protocol 2
  HostKeyAlgorithms ssh-rsa,ssh-dss
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Adjust docker hostname

Compose

Code Block
themeRDark
services:
  gerrit:
    image: gerritcodereview/gerrit
    hostname: gerrit2.ons.zone

Kubernetes

Add ssh admin key

Replication

https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md

fixed in https://www.gerritcodereview.com/2.14.html

Replication Use Case - commit change

Make change on gerrit, merge, kick off replication job, view change on gerrit2

Midnight
ubuntu@ip-172-31-27-4:~$ sudo helm install --name pgstg stable/postgresql
NAME:   pgstg
LAST DEPLOYED: Sat Apr 27 21:51:58 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Secret
NAME              TYPE    DATA  AGE
pgstg-postgresql  Opaque  1     0s
==> v1/Service
NAME                       TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)   AGE
pgstg-postgresql-headless  ClusterIP  None           <none>       5432/TCP  0s
pgstg-postgresql           ClusterIP  10.43.163.107  <none>       5432/TCP  0s
==> v1beta2/StatefulSet
NAME              DESIRED  CURRENT  AGE
pgstg-postgresql  1        1        0s
==> v1/Pod(related)
NAME                READY  STATUS   RESTARTS  AGE
pgstg-postgresql-0  0/1    Pending  0         0s


NOTES:
** Please be patient while the chart is being deployed **
PostgreSQL can be accessed via port 5432 on the following DNS name from within your cluster:
    pgstg-postgresql.default.svc.cluster.local - Read/Write connection
To get the password for "postgres" run:
    export POSTGRES_PASSWORD=$(kubectl get secret --namespace default pgstg-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)
To connect to your database run the following command:
    kubectl run pgstg-postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:10.7.0 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host pgstg-postgresql -U postgres


To connect to your database from outside the cluster execute the following commands:
    kubectl port-forward --namespace default svc/pgstg-postgresql 5432:5432 &
    PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres

describe pod shows
  Warning  FailedScheduling  21s (x6 over 3m42s)  default-scheduler  pod has unbound immediate PersistentVolumeClaims
Workaround:
Modify underlying yaml files to use a persistent volume with ReadWriteMany access



K8s only

it clone https://github.com/helm/charts.git

sudo helm install postgresql --name pg

ubuntu@ip-172-31-27-4:~/charts/stable$ helm delete pg
release "pg" deleted
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pv --all-namespaces
No resources found.
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pvc --all-namespaces
NAMESPACE   NAME                              STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
default     data-pg-postgresql-0              Pending                                                     4m48s
default     data-pgstg-postgresql-0           Pending                                                     14h
default     data-wishful-skunk-postgresql-0   Pending                                                     13m

ubuntu@ip-172-31-27-4:~/charts/stable$ vi pg-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-serv-prov-nfs-server-provisioner-0
spec:
  capacity:
    storage: 200Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /srv/volumes/nfs-serv-prov-nfs-server-provisioner-0
  claimRef:
    namespace: kube-system
    name: nfs-serv-prov-nfs-server-provisioner-0


ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl apply -f pg-pv.yaml 
persistentvolume/nfs-serv-prov-nfs-server-provisioner-0 created


ubuntu@ip-172-31-27-4:~/charts/stable$ helm delete --purge pg
release "pg" deleted

sudo helm install postgresql --name pg

ubuntu@ip-172-31-27-4:~/charts/stable$ helm delete --purge pg
release "pg" deleted
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pvc --all-namespaces
NAMESPACE   NAME                   STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
default     data-pg-postgresql-0   Pending                                                     7m23s
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl delete pvc data-pg-postgresql-0
persistentvolumeclaim "data-pg-postgresql-0" deleted


change storage-class from - to nfs-provisioner
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pvc --all-namespaces
NAMESPACE   NAME                   STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS      AGE
default     data-pg-postgresql-0   Pending                                      nfs-provisioner   7s


follow
https://severalnines.com/blog/using-kubernetes-deploy-postgresql


ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl create -f postgres-storage.yaml 
persistentvolume/postgres-pv-volume created
persistentvolumeclaim/postgres-pv-claim created
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pv --all-namespaces
NAME                                     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                                STORAGECLASS   REASON   AGE
nfs-serv-prov-nfs-server-provisioner-0   200Gi      RWO            Retain           Available   kube-system/nfs-serv-prov-nfs-server-provisioner-0                           10m
postgres-pv-volume                       5Gi        RWX            Retain           Bound       default/postgres-pv-claim                            manual                  23s
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get pvc --all-namespaces
NAMESPACE   NAME                STATUS   VOLUME               CAPACITY   ACCESS MODES   STORAGECLASS   AGE
default     postgres-pv-claim   Bound    postgres-pv-volume   5Gi        RWX            manual         32s




ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl create -f postgres-deployment.yaml 
deployment.extensions/postgres created


ubuntu@ip-172-31-27-4:~$ kubectl get pods --all-namespaces
NAMESPACE       NAME                                      READY   STATUS      RESTARTS   AGE
default         nfs-serv-prov-nfs-server-provisioner-0    1/1     Running     0          26m
default         postgres-78f78bfbfc-pw4zp                 1/1     Running     0          22s


ubuntu@ip-172-31-27-4:~/charts/stable$ vi postgres-service.yaml
ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl create -f postgres-service.yaml 
service/postgres created


ubuntu@ip-172-31-27-4:~/charts/stable$ kubectl get svc postgres
NAME       TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
postgres   NodePort   10.43.57.215   <none>        5432:30170/TCP   25s


ubuntu@ip-172-31-27-4:~/charts/stable$ sudo apt install postgresql-client-common
ubuntu@ip-172-31-27-4:~/charts/stable$ sudo apt-get install postgresql-client

ubuntu@ip-172-31-27-4:~/charts/stable$ psql -h localhost -U postgresadmin --password -p 30170 postgresdb
Password for user postgresadmin: 
psql (10.7 (Ubuntu 10.7-0ubuntu0.18.04.1), server 10.4 (Debian 10.4-2.pgdg90+1))
Type "help" for help.


postgresdb-# \l
                                 List of databases
    Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
------------+----------+----------+------------+------------+-----------------------
 postgres   | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 postgresdb | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
            |          |          |            |            | postgres=CTc/postgres
 template1  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
            |          |          |            |            | postgres=CTc/postgres
(4 rows)


dump
ubuntu@ip-172-31-27-4:~/charts/stable$ pg_dump -h localhost -U postgresadmin -p 30170 -W -F t postgresdb
Password: 
woc.dat0000600 0004000 0002000 00000003034 13461323457 0014447 0ustar00postgrespostgres0000000 0000000 PGDMP
postgresdb10.4 (Debian 10.4-2.pgdg90+1)#10.7 (Ubuntu 10.7-0ubuntu0.18.04.1)
                                                                          0ENCODINENCODINGSET client_encoding = 'UTF8';
false
     0
STDSTRINGS
STDSTRINGS(SET standard_conforming_strings = 'on';
false
     00
SEARCHPATH
SEARCHPATH8SELECT pg_catalog.set_config('search_path', '', false);
false
     126216384
postgresdDATABASEzCREATE DATABASE postgresdb WITH TEMPLATE = template0 ENCODING = 'UTF8' LC_COLLATE = 'en_US.utf8' LC_CTYPE = 'en_US.utf8';
DROP DATABASE postgresdb;
postgresfalse26152200publicSCHEMACREATE SCHEMA public;
DROP SCHEMA public;
postgresfalse
SCHEMA publicCOMMENT6COMMENT ON SCHEMA public IS 'standard public schema';
postgresfalse3307912980plpgsql	EXTENSION?CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;
DROP EXTENSION plpgsql;
false
     00EXTENSION plpgsqlCOMMENT@COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';
false1restore.sql0000600 0004000 0002000 00000002333 13461323457 0015375 0ustar00postgrespostgres0000000 0000000 --
-- NOTE:
--
-- File paths need to be edited. Search for $$PATH$$ and
-- replace it with the path to the directory containing
-- the extracted data files.
--
--
-- PostgreSQL database dump
--

-- Dumped from database version 10.4 (Debian 10.4-2.pgdg90+1)
-- Dumped by pg_dump version 10.7 (Ubuntu 10.7-0ubuntu0.18.04.1)

SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET client_min_messages = warning;
SET row_security = off;

DROP EXTENSION plpgsql;
DROP SCHEMA public;
--
-- Name: public; Type: SCHEMA; Schema: -; Owner: postgres
--

CREATE SCHEMA public;


ALTER SCHEMA public OWNER TO postgres;

--
-- Name: SCHEMA public; Type: COMMENT; Schema: -; Owner: postgres
--

COMMENT ON SCHEMA public IS 'standard public schema';


--
-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner: 
--

CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;


--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner: 
--

COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';


--
-- PostgreSQL database dump complete
--


Backup

Restore

Gerrit

Config Jobs

login as default admin

create test repo

Gerrit repo url

Code Block
themeRDark
verified
git clone ssh://admin@gerrit2.ons.zone:29418/test.git
remote: Counting objects: 2, done
git clone admin@gerrit2.ons.zone:29418/test.git
admin@gerrit2.ons.zone: Permission denied (publickey).

verified on gerrit3 to 2 host
ubuntu@ip-172-31-31-191:~$ git clone  ssh://admin@gerrit2.ons.zone:29418/replicate.git
Cloning into 'replicate'...

verified admin user
ubuntu@ip-172-31-31-191:~$ ssh -p 29418 admin@gerrit2.ons.zone
Warning: Permanently added '[gerrit2.ons.zone]:29418,[3.17.20.86]:29418' (RSA) to the list of known hosts.
  Hi Administrator, you have successfully connected over SSH.


gerrit source - /var/gerrit/etc/gerrit.config

Code Block
themeRDark
# adjust server name
[gerrit]
        #canonicalWebUrl = http://fcdbe931c71d
        canonicalWebUrl = http://gerrit2.ons.zone

[receive]
        #enableSignedPush = false
        enableSignedPush = truef


gerrit source - /var/gerrit/etc/replication.config

Code Block
themeRDark
# ssh version
[remote "gerrit2"]
  url = ssh://admin@gerrit2.ons.zone:29418/${name}.git
  push = +refs/heads/*:refs/heads/*
  push = +refs/tags/*:refs/tags/*


# http version
[gerrit]
  defaultForceUpdate = true
[remote "gerrit2"]
  url = http://admin:NobJjm7wlDFvAObPWo5ZwlnmQEwdt9fyBJlJdIE5WQ@gerrit2.ons.zone:8080/${name}.git
  mirror = true
  threads = 3
  push = +refs/heads/*:refs/heads/*
  push = +refs/changes/*:refs/changes/*
  push = +refs/tags/*:refs/tags/*

Host ~/.ssh/config

Code Block
themeMidnight
Host remote-alias gerrit2.ons.zone
  IdentityFile ~/.ssh/onap_rsa
  Hostname gerrit2.ons.zone
  Protocol 2
  HostKeyAlgorithms ssh-rsa,ssh-dss
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Adjust docker hostname

Compose

Code Block
themeRDark
services:
  gerrit:
    image: gerritcodereview/gerrit
    hostname: gerrit2.ons.zone


Kubernetes

Add ssh admin key


Verification

verify port 

 curl http://gerrit.ons.zone:8080/ssh_info

Replication

https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md

fixed in https://www.gerritcodereview.com/2.14.html

Replication Use Case - commit change

Make change on gerrit, merge, kick off replication job, view change on gerrit2

Code Block
themeRDark
# 3 machines
# obriensystems dev laptop
# gerrit source server
# gerrit2 replication server
# on remote dev host - against gerrit
git clone "ssh://admin@gerrit.ons.zone:29418/test" && scp -p -P 29418 admin@gerrit.ons.zone:hooks/commit-msg "test/.git/hooks/"
cd test/
vi test.sh 
git add test.sh 
git commit -s --amend
git review
# getting merge conflict - needed to remove old commit id
vi test.sh 
git add test.sh 
git rebase --continue
git review

# move to gerrit UI, +2 review, merge
# on gerrit server
ssh ubuntu@gerrit.ons.zone
# tail the logs to the gerrit container

# on dev laptop
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone gerrit plugin reload replication
[2019-03-28 15:25:57,246] [SSH gerrit plugin reload replication (admin)] INFO  com.google.gerrit.server.plugins.PluginLoader : Reloaded plugin replication, version v2.16.6
obrienbiometrics:test michaelobrien$ ssh -p 29418 admin@gerrit.ons.zone replication list
Remote: gerrit2
Url: admin@gerrit2.ons.zone:8080/${name}.git

[2019-03-28 15:26:57,963] [WorkQueue-1] INFO  com.google.gerrit.server.plugins.CleanupHandle : Cleaned plugin plugin_replication_190328_0446_6094540689096397413.jar
# debug on
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set DEBUG                                          
# debug off
ssh -p 29418 admin@gerrit.ons.zone gerrit logging set reset
ssh -p 29418 admin@gerrit.ons.zone replication start --wait --all 
# nothing yet - debugging container I only see a recent
var/gerrit/data/replication/ref-updates
-rw-r--r-- 1 gerrit gerrit   45 Mar 28 15:25 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066
bash-4.2$ cat 9cbb43eb3ce03badc8b3c7dc52ef84d8d6e67066 
{"project":"test","ref":"refs/heads/master"}


Issue was the key - after changing the url to 
url = admin@gerrit2.ons.zone:29418/${name}.git
I can ssh directly from gerrit to gerrit2 but they key is n/a for the container yet
sshd_log
[2019-03-28 15:57:50,164 +0000] b2bd0870 admin a/1000000 replication.start.--all 3ms 1ms 0
replication_log
[2019-03-28 17:34:07,816] [72da30d3] Replication to admin@gerrit2.ons.zone:29418/All-Users.git started...
[2019-03-28 17:34:07,834] [72da30d3] Cannot replicate to admin@gerrit2.ons.zone:29418/All-Users.git
org.eclipse.jgit.errors.TransportException: admin@gerrit2.ons.zone:29418/All-Users.git: reject HostKey: gerrit2.ons.zone
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:192)


# I am running hashed
ubuntu@ip-172-31-15-176:~$ grep "HashKnownHosts" /etc/ssh/ssh_config
    HashKnownHosts yes


# tried - it may be my url
Url: ssh://admin@gerrit2.ons.zone:29418/${name}.git
from
Url: admin@gerrit2.ons.zone:29418/${name}.git

[2019-03-28 21:54:04,089] [] Canceled 3 replication events during shutdown
[2019-03-28 21:54:17,738] [] scheduling replication All-Projects:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
[2019-03-28 21:54:17,750] [] scheduled All-Projects:..all.. => [283d568e] push ssh://admin@gerrit2.ons.zone:29418/All-Projects.git to run after 15s
[2019-03-28 21:54:17,750] [] scheduling replication All-Users:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Users.git
[2019-03-28 21:54:17,751] [] scheduled All-Users:..all.. => [684a6e1d] push ssh://admin@gerrit2.ons.zone:29418/All-Users.git to run after 15s
[2019-03-28 21:54:17,751] [] scheduling replication test:..all.. => ssh://admin@gerrit2.ons.zone:29418/test.git
[2019-03-28 21:54:17,751] [] scheduled test:..all.. => [a84066fe] push ssh://admin@gerrit2.ons.zone:29418/test.git to run after 15s
[2019-03-28 21:54:32,751] [283d568e] Replication to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git started...
[2019-03-28 21:54:32,857] [283d568e] Cannot replicate to ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
org.eclipse.jgit.errors.TransportException: ssh://admin@gerrit2.ons.zone:29418/All-Projects.git: reject HostKey: gerrit2.ons.zone



Hostname is generated by docker - overriding
using hostname not hostname.domainname pair - as gerrit will only pick up the prefix
services:
  gerrit:
    image: gerritcodereview/gerrit
    hostname: gerrit2.ons.zone

ubuntu@ip-172-31-6-115:~$ sudo docker exec -it ubuntu_gerrit_1 bash
bash-4.2$ hostname
gerrit2.ons.zone


#trying protocol 1 (insecure) instead of 2
/home/ubuntu/.ssh/config line 5: Bad protocol spec '1'.


Verifying via https://gerrit.googlesource.com/plugins/replication/+/master/src/main/resources/Documentation/config.md
PreferredAuthentications publickey




Working with Igor - trying http instead of ssh






p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}



2019-03-29 19:54:23,740] [6737289e] Cannot replicate replicate; Remote repository error: http://admin@gerrit2.ons.zone:8080/replicate.git: replicate unavailable

[2019-03-29 19:54:23,741] [a71c401e] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...

[2019-03-29 19:54:23,754] [e7c79889] Replication to http://admin@gerrit2.ons.zone:8080/All-Projects.git completed in 121ms, 15001ms delay, 0 retries

[2019-03-29 19:54:23,756] [273130a7] Replication to http://admin@gerrit2.ons.zone:8080/All-Users.git completed in 116ms, 15000ms delay, 0 retries

[2019-03-29 19:54:23,762] [a71c401e] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...e9efbebcb130387cfa65e6f3c47dea5d005f8bbe, srcRef=refs/heads/master, forceUpdate, message=null]]

[2019-03-29 19:54:24,093] [a71c401e] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/test.git, reason: prohibited by Gerrit: not permitted: force update

[2019-03-29 19:54:24,094] [a71c401e] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 352ms, 15103ms delay, 0 retries




changed signed to true








p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}



[2019-03-29 20:11:48,466] [a722008c] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...

[2019-03-29 20:11:48,467] [a722008c] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...a9f165b560889e937a10ac45f425c6d727a8fb78, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/changes/03/1003/1, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/changes/03/1003/1, forceUpdate, message=null]]

[2019-03-29 20:11:48,662] [a722008c] Failed replicate of refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes

[2019-03-29 20:11:48,662] [a722008c] Failed replicate of refs/changes/03/1003/1 to http://admin@gerrit2.ons.zone:8080/test.git, reason: cannot combine normal pushes and magic pushes

[2019-03-29 20:11:48,663] [a722008c] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 197ms, 15000ms delay, 0 retries






review+2/committed




p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}



[2019-03-29 20:15:12,674] [a7d4407f] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...
[2019-03-29 20:15:12,676] [a7d4407f] Push to http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...60d083b4f4b0916a6dc0da694c52a5e7ff08a9b7, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/heads/master, forceUpdate, message=null]]
[2019-03-29 20:15:12,862] [a7d4407f] Failed replicate of refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:8080/test.git, reason: NoteDb update requires -o notedb=allow
[2019-03-29 20:15:12,862] [a7d4407f] Failed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:8080/test.git, reason: prohibited by Gerrit: not permitted: force update
[2019-03-29 20:15:12,863] [a7d4407f] Replication to http://admin@gerrit2.ons.zone:8080/test.git completed in 188ms, 15000ms delay, 0 retries




gerrit.defaultForceUpdate :	If true, the default push refspec will be set to use forced update to the remote when no refspec is given. By default, false.


[gerrit]
        defaultForceUpdate = true


still magic error - added above to target gerrit



https://groups.google.com/forum/#!msg/repo-discuss/m2E72F2oiuo/w-lWg0WUZYIJ




p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Menlo; color: #ffffff; background-color: #132a65}
span.s1 {font-variant-ligatures: no-common-ligatures}



[[A[2019-03-29 21:03:24,045] [] scheduling replication All-Projects:..all.. => http://admin@gerrit2.ons.zone:8080/All-Projects.git

[2019-03-29 21:03:24,068] [] scheduled All-Projects:..all.. => [71c8678f] push http://admin@gerrit2.ons.zone:8080/All-Projects.git to run after 15s

[2019-03-29 21:03:24,071] [] scheduling replication All-Users:..all.. => http://admin@gerrit2.ons.zone:8080/All-Users.git

[2019-03-29 21:03:24,072] [] scheduled All-Users:..all.. => [b1cedf9b] push http://admin@gerrit2.ons.zone:8080/All-Users.git to run after 15s

[2019-03-29 21:03:24,075] [] scheduling replication replicate:..all.. => http://admin@gerrit2.ons.zone:8080/replicate.git

[2019-03-29 21:03:24,076] [] scheduled replicate:..all.. => [f19bf7a7] push http://admin@gerrit2.ons.zone:8080/replicate.git to run after 15s

[2019-03-29 21:03:24,079] [] scheduling replication test:..all.. => http://admin@gerrit2.ons.zone:8080/test.git

[2019-03-29 21:03:24,080] [] scheduled test:..all.. => [3192ef91] push http://admin@gerrit2.ons.zone:8080/test.git to run after 15s

[2019-03-29 21:03:39,070] [71c8678f] Replication to http://admin@gerrit2.ons.zone:8080/All-Projects.git started...

[2019-03-29 21:03:39,072] [b1cedf9b] Replication to http://admin@gerrit2.ons.zone:8080/All-Users.git started...

[2019-03-2829 1721:3403:0739,834076] [72da30d3f19bf7a7] Cannot replicateReplication to http://admin@gerrit2.ons.zone:294188080/All-Usersreplicate.git orgstarted.eclipse.jgit.errors.TransportException: .

[2019-03-29 21:03:39,245] [71c8678f] Replication to http://admin@gerrit2.ons.zone:294188080/All-UsersProjects.git: reject HostKey: gerrit2.ons.zone
	at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:192)


# I am running hashed
ubuntu@ip-172-31-15-176:~$ grep "HashKnownHosts" /etc/ssh/ssh_config
    HashKnownHosts yes


# tried - it may be my url
Url: ssh completed in 173ms, 15004ms delay, 0 retries

[2019-03-29 21:03:39,246] [3192ef91] Replication to http://admin@gerrit2.ons.zone:8080/test.git started...

[2019-03-29 21:03:39,245] [f19bf7a7] Push to http://admin@gerrit2.ons.zone:8080/replicate.git references: [RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...410858cdd130ee2d56700d199f4021c246c1d22b, srcRef=refs/heads/master, forceUpdate, message=null]]

[2019-03-29 21:03:39,245] [b1cedf9b] Replication to http://admin@gerrit2.ons.zone:29418/${name}.git
from
Url: admin@gerrit2.ons.zone:29418/${name}.git8080/All-Users.git completed in 173ms, 15000ms delay, 0 retries

[2019-03-2829 21:5403:0439,089281] [3192ef91] CanceledPush 3 replication events during shutdown
[2019-03-28 21:54:17,738] [] scheduling replication All-Projects:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Projects.git
[2019-03-28 21:54:17,750] [] scheduled All-Projects:..all.. => [283d568e] push ssh://admin@gerrit2.ons.zone:29418/All-Projects.git to run after 15s
[2019-03-28 21:54:17,750] [] scheduling replication All-Users:..all.. => ssh://admin@gerrit2.ons.zone:29418/All-Users.git
[2019-03-28 21:54:17,751] [] scheduled All-Users:..all.. => [684a6e1d] push sshto http://admin@gerrit2.ons.zone:8080/test.git references: [RemoteRefUpdate[remoteName=refs/changes/03/1003/1, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/changes/03/1003/1, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/changes/03/1003/meta, NOT_ATTEMPTED, (null)...60d083b4f4b0916a6dc0da694c52a5e7ff08a9b7, srcRef=refs/changes/03/1003/meta, forceUpdate, message=null], RemoteRefUpdate[remoteName=refs/heads/master, NOT_ATTEMPTED, (null)...477762111d6ad43984fd3ee908730267880469c2, srcRef=refs/heads/master, forceUpdate, message=null]]

[2019-03-29 21:03:39,682] [3192ef91] Failed replicate of refs/changes/03/1003/1 to http://admin@gerrit2.ons.zone:294188080/All-Userstest.git to run after 15s, reason: cannot combine normal pushes and magic pushes

[2019-03-2829 21:5403:1739,751682] [3192ef91] Failed schedulingreplicate replication test:..all.. => sshof refs/changes/03/1003/meta to http://admin@gerrit2.ons.zone:294188080/test.git, reason: cannot combine normal pushes and magic pushes

[2019-03-2829 21:5403:1739,751682] [3192ef91] scheduled test:..all.. => [a84066fe] push sshFailed replicate of refs/heads/master to http://admin@gerrit2.ons.zone:294188080/test.git to run after 15s, reason: cannot combine normal pushes and magic pushes

[2019-03-2829 21:5403:3239,751682] [283d568e3192ef91] Replication to sshhttp://admin@gerrit2.ons.zone:29418/All-Projects.git started...8080/test.git completed in 435ms, 15166ms delay, 0 retries

[2019-03-2829 21:5403:3239,857704] [283d568ef19bf7a7] CannotFailed replicate toof ssh:refs//admin@gerrit2.ons.zone:29418/All-Projects.git
org.eclipse.jgit.errors.TransportException: sshheads/master to http://admin@gerrit2.ons.zone:294188080/All-Projectsreplicate.git:, reject HostKeyreason: gerrit2.ons.zoneprohibited by Gerrit: not Hostnamepermitted: isforce generatedupdate
by
docker - overriding
using hostname not hostname.domainname pair - as gerrit will only pick up the prefix
services:
  gerrit:
    image: gerritcodereview/gerrit
    hostname: gerrit2.ons.zone

ubuntu@ip-172-31-6-115:~$ sudo docker exec -it ubuntu_gerrit_1 bash
bash-4.2$ hostname
gerrit2.ons.zone[2019-03-29 21:03:39,705] [f19bf7a7] Replication to http://admin@gerrit2.ons.zone:8080/replicate.git completed in 628ms, 15000ms delay, 0 retries

todo: need adminURL to create repos

JSCH - Java Secure Channel - issue with sha2 known_hosts entries 

A video of what is being fixed

View file
name20190329_gerrit_replication_start_with-jsch-issue.mp4
height250

View file
name20190412_gerrit3_replication_partial_zoom_0.mp4
height250

According to https://groups.google.com/forum/#!topic/repo-discuss/9PTfVG8vdAU for https://github.com/eclipse/jgit/blob/master/org.eclipse.jgit/src/org/eclipse/jgit/transport/JschConfigSessionFactory.java#L191

...