Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Openstack Adapter call to MultiCloud fails

Description

Testing vf-Module creation via MultiCloud Openstack Adapter, the call from
SO-Openstack-Adapter to MultiCloud fails.
Test executed:

  1. Successfull execution of vf-module creation via so-openstack-adapter

  2. Changed cloud_sites DB entry "ORCHESTRATOR" value to "multicloud" and "VNF Adapter Rest API endpoint" version version “v2”, restart of SO-BPMN pod

  3. Re-execution of vf-module creation

Result:

  • so-openstack-adapter tries to call multicloud

  • call fails (most likely due to certificate error)

Call (from debug.log):

Sending HTTP GET to:https://msb-iag.onap:443/api/multicloud/v1/DT/RegionOne/infra_workload?name=Python_ONAP_SDK_vf_module_instance_5e7c7a79-6 c55-46c7-99ee-c0e8c80bf636 with request headers:{X-RequestID=[a7f95ea6-2476-4131-b18b-3c6a08909ffb], Project=[b5f12d2cdda0449194 8d9fb4cb00994e], X-ONAP-PartnerName=[UNKNOWN], Accept=[application/json], X-InvocationID=[5cbf1df3-2ee8-48cc-a784-e6ef32062ea2], X-ECOMP-RequestID=[a7f95ea6-2476-4131-b18b-3c6a08909ffb], X-TransactionID=[a7f95ea6-2476-4131-b18b-3c6a08909ffb], X-ONAP-Reques tID=[a7f95ea6-2476-4131-b18b-3c6a08909ffb]}

Fails with:

2020-05-13T09:36:31.644Z|a7f95ea6-2476-4131-b18b-3c6a08909ffb|org.apache.cxf.phase.PhaseInterceptorChain - Interceptor for {http2020-05-13T09:36:31.644Z|a7f95ea6-2476-4131-b18b-3c6a08909ffb|org.apache.cxf.phase.PhaseInterceptorChain - Interceptor for {https://msb-iag.onap:443/api/multicloud/v1/DT/RegionOne/infra_workload}WebClient has thrown exception, unwinding noworg.apache.cxf.interceptor.Fault: Could not send Message. at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:67) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:710) at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1050) at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:897) at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:866) at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:431) at org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:135) at org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:130) at org.apache.cxf.jaxrs.client.SyncInvokerImpl.get(SyncInvokerImpl.java:50) at org.apache.cxf.jaxrs.client.spec.InvocationBuilderImpl.get(InvocationBuilderImpl.java:88) at org.onap.so.client.RestRequest.get(RestRequest.java:52) at org.onap.so.client.RestClient.buildRequest(RestClient.java:292) at org.onap.so.client.RestClient.lambda$method$3(RestClient.java:288) at net.jodah.failsafe.Functions.lambda$resultSupplierOf$11(Functions.java:283) at net.jodah.failsafe.internal.executor.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:67) at net.jodah.failsafe.Execution.executeSync(Execution.java:117) at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:319) at net.jodah.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:71) at org.onap.so.client.RestClient.method(RestClient.java:288) at org.onap.so.client.RestClient.get(RestClient.java:223) at org.onap.so.openstack.utils.MsoMulticloudUtils.queryStack(MsoMulticloudUtils.java:290) at org.onap.so.openstack.utils.MsoMulticloudUtils.queryVdu(MsoMulticloudUtils.java:914) at org.onap.so.adapters.vnf.MsoVnfPluginAdapterImpl.deleteVfModule(MsoVnfPluginAdapterImpl.java:1089) at org.onap.so.adapters.vnf.MsoVnfPluginAdapterImpl$$FastClassBySpringCGLIB$$2ff25d29.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at org.onap.so.adapters.vnf.MsoVnfPluginAdapterImpl$$EnhancerBySpringCGLIB$$6dc9ba0e.deleteVfModule(<generated>) at org.onap.so.adapters.vnf.VnfAdapterRestV2$DeleteVfModuleTask.run(VnfAdapterRestV2.java:197) at java.lang.Thread.run(Thread.java:748)Caused by: java.io.IOException: IOException invoking https://msb-iag.onap:443/api/multicloud/v1/DT/RegionOne/infra_workload?name=Python_ONAP_SDK_vf_module_instance_5e7c7a79-6c55-46c7-99ee-c0e8c80bf636: HTTPS hostname wrong:  should be <msb-iag.onap> at sun.reflect.GeneratedConstructorAccessor210.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1402) at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1386) at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)

When trying to execute wget on the so-openstack-adapter container:

/app/logs/openstack $ wget https://msb-iag.onap:443/api/multicloud/v1/DT/RegionOne Connecting to msb-iag.onap:443 (10.43.99.138:443) ssl_client: msb-iag.onap: certificate verification failed: unable to get local issuer certificate wget: error getting response: Connection reset by peer

Most likely the Certificate-Validation has to be disabled

 

 

Environment

OOM Daily Master deployment (DT Labs) from 13.05.2020

Activity

Show:

Andreas Geißler August 10, 2020 at 12:55 PM

Hi,
sorry, I found the request to check this Fix quite late.
I tried to retest the usecase (activation of MultiCloud and vfmodule creation) on the actual Frankfur release.
I updated the catalogdb cloud_sites table, changed the API version to v2 and started the test again.

The test does not fail, but the reason was, that it did not use the MultiCloud at all!!

The SO Openstack Adapter seems to ignore completely the "ORCHESTRATOR" entry in the DB.

I will check, whether there are new "Add a Cloud" instructions, otherwise I will raise a new bug.
Best regards
Andreas

Former user May 29, 2020 at 4:33 PM

Team - is on leave. I am closing this ticket for now but be ready to jump if Andreas will find another issue.

Former user May 27, 2020 at 7:30 AM

Hi , ,   - all relevant patches have been merged to Frankfurt oom today (and to master some time ago), so since now, this issue should no longer be visible on today's or newer environments.

Seshu Kumar Mudiganti May 26, 2020 at 5:54 AM

: Could you please confirm this...

Former user May 21, 2020 at 6:58 PM
Edited

Ok, I have brought up a new ONAP env in Windriver OOF project as of 5/20/2020.  It is running SO 1.6.1.

In short, with the two oom fixes below, the so-msb-multicloud communication is working.

 

Detailed steps:

After onap installed, I cherrypicked  's two oom patches 

 https://gerrit.onap.org/r/c/oom/+/107649/3 (Set certificate SAN as msb-iag) and https://gerrit.onap.org/r/c/oom/+/106826/3 ([so][msb] Update MSB url to match with its certificate) 

then helm deleted so and msb

I also editted 

so/charts/so-bpmn-infra/resources/config/overrides/override.yaml

to set the vnf endpoint to v2 (to avoid later configmap edit step)

 

then:

  • make so msb

  • make onap

  • helm deploy dev local/onap -f /root/oom/kubernetes/onap/resources/environments/public-cloud.yaml -f /root/integration-override.yaml --namespace onap

Then:

  • run 'demo init' script to set up some data

  • added a cloud site

  • from vid - made a service instance, vnf, sndc preload

  • from vid - make a vfw packet generator vfmodule

  • key point - so openstack adapter is able to communicate to multicloud via msb with no other workarounds (besides two oof patches above)

    • (I did not concern myself with complete / correct deployment of the vfmodule as that takes more time)

Snippet from openstack adatper debug.log showing successful communication to multicloud via msb:

2020-05-21T18:41:39.561Z|f95b332f-8bec-4a00-899e-618783f48ded|o.o.l.filter.spring.SpringClientPayloadFilter - =======================response end=================================================
2020-05-21T18:41:39.563Z|f95b332f-8bec-4a00-899e-618783f48ded|org.onap.so.openstack.utils.MsoMulticloudUtils - Query multicloud HEAT stack: vfw-vfm-pkg-2 in tenant 709ba629fe194f8699b12f9d6ffd86a0
2020-05-21T18:41:39.566Z|f95b332f-8bec-4a00-899e-618783f48ded|org.onap.so.openstack.utils.MsoMulticloudUtils - Multicloud Endpoint is: https://msb-iag:443/api/multicloud/v1/CloudOwner/ONAP-POD-01-Rail-06/infra_workload?name=vfw-vfm-pkg-2
2020-05-21T18:41:39.598Z|f95b332f-8bec-4a00-899e-618783f48ded|o.o.logging.filter.base.PayloadLoggingClientFilter - Sending HTTP GET to:https://msb-iag:443/api/multicloud/v1/CloudOwner/ONAP-POD-01-Rail-06/infra_workload?name=vfw-vfm-pkg-2 with request headers:{X-RequestID=[f95b332f-8bec-4a00-899e-618783f48ded], Project=[709ba629fe194f8699b12f9d6ffd86a0], X-ONAP-PartnerName=[UNKNOWN], Accept=[application/json], X-InvocationID=[0dcbcb98-759b-466d-9b47-109caa1740f3], X-ECOMP-RequestID=[f95b332f-8bec-4a00-899e-618783f48ded], X-TransactionID=[f95b332f-8bec-4a00-899e-618783f48ded], X-ONAP-RequestID=[f95b332f-8bec-4a00-899e-618783f48ded]}
2020-05-21T18:41:40.561Z|f95b332f-8bec-4a00-899e-618783f48ded|o.o.logging.filter.base.PayloadLoggingClientFilter - Response from method:GET performed on uri:https://msb-iag:443/api/multicloud/v1/CloudOwner/ONAP-POD-01-Rail-06/infra_workload?name=vfw-vfm-pkg-2 has http status code:200 and response headers:{-content-encoding=[gzip], Allow=[GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS], connection=[keep-alive], Content-Length=[113], content-location=https://msb-iag.onap:443/api/multicloud-titaniumcloud/v1/CloudOwner/ONAP-POD-01-Rail-06/infra_workload?name=vfw-vfm-pkg-2, content-type=[application/json], Date=[Thu, 21 May 2020 18:41:40 GMT], Server=[openresty], status=[200]}
2020-05-21T18:41:40.563Z|f95b332f-8bec-4a00-899e-618783f48ded|o.o.logging.filter.base.PayloadLoggingClientFilter - {"template_type":"HEAT","workload_id":"","workload_status":"GET_COMPLETE","workload_status_reason":{"stacks":[]}}

2020-05-21T18:41:40.565Z|f95b332f-8bec-4a00-899e-618783f48ded|org.onap.so.openstack.utils.MsoMulticloudUtils - Multicloud GET Response: org.apache.cxf.jaxrs.impl.ResponseImpl@f87bbe8
2020-05-21T18:41:40.583Z|f95b332f-8bec-4a00-899e-618783f48ded|org.onap.so.openstack.utils.MsoMulticloudUtils - Multicloud Create Response Body: org.onap.so.openstack.utils.MulticloudQueryResponse@6bfbb50[templateType=HEAT,workloadId=,workloadStatus=GET_COMPLETE,workloadStatusReason={"stacks":[]}]
2020-05-21T18:41:40.601Z|f95b332f-8bec-4a00-899e-618783f48ded|org.onap.so.openstack.utils.MsoMulticloudUtils - Multicloud getWorkloadStack() returning Stack Object: Stack{description='null', links=null, stackStatusReason='null', stackName='null', updatedTime=null, creationTime=null, stackStatus='NOT_FOUND', id='null', outputs='null', parameters='{}', files='null'}

Done

Details

Assignee

Reporter

Labels

Fix versions

Affects versions

Priority

Created May 13, 2020 at 10:07 AM
Updated August 10, 2020 at 12:55 PM
Resolved May 29, 2020 at 4:33 PM

Flag notifications