2024-08-06 Security Subcommittee Meeting Notes

NOTE:  No high priority findings this scan, everything looks good from a license perspective.

ONAP Project Scan and Findings - June 2024: https://lfscanning.org/reports/onap/onap-2024-06-0592bdfc-78a5-4fa8-a9fd-a7de581457ec.html

Atlanta, Georgia from October 22-23

Schedule: https://email.linuxfoundation.org/e3t/Ctc/RI+113/cZw--04/VW20421KRWSMW25H9dF35S2gmVwDG2D5j9vZYN70SktR3qgyTW8wLKSR6lZ3msW6f-nGc8jMw2WW6PLn8F5XylNbW5kHdzx1pMHtfW2ldjmB6-R-7zVq9SV73CRtNtW3VN9Gq8wcwlWW4Fn7pT1PzCz_W32P4RN40kW2bW2VgnfM63lsqhN4RkJdDg7q38W4G_b2n1WP-XGW2F4z262Xn7F1W76GYT18zQGYZW8jj8h993XpPdW2kWrbS5PFXd9N86SL4hfqQG9N4hp9_Qz682JW1QPNg577TNNXW44YFbY2-2hwvW8BdZw75VVmXPN5ZdyPZ-8_nkW7ctCB37wg7yGW53chgq2lb9t6Vgg5sm37sNb8W4QdqX16qwWQCW7ld23P8qQgtVW3Yb-Bm1qr2p3N53g5VPf849qf1-JnKn04

Registration: 

https://email.linuxfoundation.org/e3t/Ctc/RI+113/cZw--04/VW20421KRWSMW25H9dF35S2gmVwDG2D5j9vZYN70Sktx3qgyTW7Y8-PT6lZ3lkW45CttV8xzKzMW8yq06x86dZvCW2ZY_J97ykfKdW4mlsVn7twbjNW3f0vrT2y-BcXV8dkBl29ZQptW4MmWwR7s8RPTW3PZgTh1J3J1wW7Sb0pj4gTvc2W7WTqDG7ppkY3W7PQzrt1ZcKzVW7-szSQ33Jfs5W8J_55Z6yGxVyW7ZzCXh7TV5C1W2W8cqn7fyyHbW2NDp1c4vw7xgW1syX951_FkwWW7c53XW8x_44KW4J1Fdx7KJyBvW7LkKhM98YtnJW5Zn8Gj3vvP0GW70JNRC2FsjkgW15lx-j3ksW5XW3TzmqM22JJtgW8WdlrM2VCmMgW3KxXtS5QgdXbf6T2YKH04

Support for Ramesh and Policy team to get gold badge.

List of people to be Policy page editors in the badging - done

Ticket opened by Tony: IT-26848 - Tony is checking on it, still has issues?

Are already available.

• Vulnerable Packages Update: The most recent reports (7/5) are on Oslo Package Updates - Security Vulnerabilities - Confluence (onap.org)
• The list shows ONAP SonarCloud reports and the most recent time a scan was run. There are some variations in dates across projects that need to be researched. Additionally, there are some deprecated project still being scanned (done).
• AAI: 10/30/2024
• CCSDK: 7/6/2024
• CPS: 7/8/2024
• DCAE: 10/30/2023
• DMaaP: 1/26/2024 - to be removed
• MSB: 10/29/2024 - to be removed
• MultiCloud: 10/30/2023
• Policy: 7/8/2024
• SDC: 7/6/2024
• SDNC: 7/6/2024
• SO: 10/30/2023
• UsecaseUI: 10/29/2024
• Remove Policy GUI in SonarCloud reports (Pawel) - https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27005
• On Amy's to do list: updating Java, Python, OS, database and utility versions (Amy) + creating Jira tickets for packages upgrades.

Upon analyzing a list of 172 projects derived from the Open Source Security Foundation
(OpenSSF) Securing Critical Projects Working Group’s List of Critical Projects,2 we observe that:
• 52% of the projects contain code written in a memory-unsafe language.
• 55% of the total lines of code (LoC) for all projects were written in a memory-unsafe language.
• The largest projects are disproportionately written in memory-unsafe languages. Of the ten largest projects by total LoC, each has a proportion of memory unsafe LoC above 26%. The median proportion using memory-unsafe languages across the ten projects is 62.5% and four of the ten project proportions exceed 94%.
• Dependency analysis of three projects written in memory-safe languages demonstrated that each one depended on other components written in memory unsafe languages.

Informative meaning.

Registration information for SBOM-a-Rama attendees and SBOM-Solutions Showcase potential exhibitors:

Register to attend SBOM-a-Rama on September 11, 2024 or the SBOM-Solutions Showcase on September 12, 2024

SBOM community meetings are held regularly.  If interested on supporting a Working Group for challenges or solutions, send an email to sbom@cisa.dhs.gov, join the meeting for just listen in or provide input to improve.

New scans iteration for packages upgrades available on the restricted Wiki, but due to instability of Nexus-IQ, all the CLM jobs were failing. Amy has opened  ticket:  IT-26799 CLM Jenkins Jobs Failing

Progress - not all jobs are failing. Pawel's access to CLM reports was lost - Jira was opened for that: https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-26825 and IT-26810.

Following the discussion at the last TSC Andreas confirmed that his tests are not failing.

Update received from Kevin on reasons CLM jobs are failing:

• dependencies errors [TO BE FIXED BY COMMUNITY]
  • https://jenkins.onap.org/view/CLM/job/ccsdk-apps-maven-clm-master/326/console 
• there are test failures [TO BE FIXED BY COMMUNITY]
  • https://jenkins.onap.org/view/CLM/job/msb-discovery-maven-clm-master/311/console
  • https://jenkins.onap.org/view/CLM/job/sdnc-apps-maven-clm-master/310/console
  • https://jenkins.onap.org/view/CLM/job/oom-platform-cert-service-maven-clm-master/210/console
• plugins missing required configuration at POM files so they fail when executed, additionally the jobs failing with this error seem abandoned as they have not run properly for a long time so they should be removed [TO BE FIXED BY COMMUNITY]:
  • https://jenkins.onap.org/view/CLM/job/vnfsdk-functest-maven-clm-master/315/console (last time it worked as expected was Oct 2021)
  • https://jenkins.onap.org/view/CLM/job/multicloud-openstack-vmware-maven-clm-master/310/console (last time it worked as expected was Nov 2022)
  • https://jenkins.onap.org/view/CLM/job/multicloud-openstack-maven-clm-master/275/console (last time it worked as expected was Sep 2022)
  • https://jenkins.onap.org/view/CLM/job/optf-has-maven-clm-master/305/console (last time it worked as expected was Jul 2023)

• nexus IQ application IDs missing: we FIXED this one by creating the missing application IDs through NexusIQ dashboard
• old clm-maven-plugin version causing incompatibility errors: we FIXED this one by removing a hardcoded version pin we had in a GlobalJJB script

• Paweł Pawlak to share info with PTLs for CCSDK and SDNC (Dan), OOM (Andreas), Multicloud (???)
• Byung-Woo Jun to open a ticket for LFN IT to remove Jenkins jobs for deprecated projects, https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-26907

Mateusz Pilat from Tata presented changes in log format for its unification. Change Request will be prepared by Mateusz. Discussion will be followed at the OOM meeting on Wednesday.

RBAC changes could be provided: Improvement for NewDelhi Release

Root access for container need was explained.

• Maggie, Justin and Andrew Lamb's input: 
  • ONAP application services will run in a namespace whose privilege and security context are restricted via Pod Security Standards Restricted policy to meet the CIS 1.6 benchmark. No workloads in this namespace will ever be permitted to run with privileges or as root within the container.
  • ONAP logging services will run in a separate namespace which allows Pod Security Standards Privileged profiles. Any workload in this namespace will be permitted to run with privileges. To mitigate the risk of such a namespace, the Kubernetes Admission Controller, via Kyverno policies, will enforce strict admission requirements on workloads in this namespace.
  • To be deployed into the privileged namespace, workloads must be running a specific container image with restrictive container startup commands and arguments, as specified in Kyverno policies written by ONAP developers. This will ensure that only ONAP SECCOM approved workloads are ever permitted to run with privileges in this namespace.
• As a more general SECCOM note, Byung - I discovered in my research that the Kyverno admission validation hook can also (1) enforce minimum versions of container images (which I wish I would have known when trying to keep log4j off my clusters last year) and (2) prevent exec operations into running containers - this would be useful in preventing an insider from accessing the privileged workload once it has already been admitted to the privileged namespace. I did not include these because they seem to be larger design requirements for cluster administrators beyond just the management of ONAP. If you think it would be within ONAP's purview to enforce these mechanisms I'm happy to include them in the next PoC iteration. - Justin
• Hope in Oslo, we can discuss/explore this further.

Further update will be discussed during Oslo. Tata Communications still plans to do some improvement for Oslo, but no detailed plan yet.

LF IT migrating CI pipeline to GitHub actions - may take to the end of fall or later, once ONAP is completed for GitHub Actions , we will do security review. Last update from Matt is that LF IT is continuously shipping one project at a time. 

4/2: in progress

At the TSC Jess mentioned Q4'24? or rather beginning of 2025.

Muddasar Ahmed presented the draft deck about LFN AI/ML use cases.

Maggie shared link:

https://www.nist.gov/itl/ai-risk-management-framework 

We need to have Ops feedback (NOC manager) on AI, what pain point could  be solved by AI.

Deck shared with Marian from Orange, feedback expected in first week of December. Under WG 11 in ORAN Alliance (doing standards for ORAN) - threat analysis will be done in the domain of AI security - OWASP TOP 10 - planned by March'24.

Runtime influence under interest.

Maggie shared the link: https://www.cisa.gov/news-events/alerts/2023/11/26/cisa-and-uk-ncsc-unveil-joint-guidelines-secure-ai-system-development 

Feedback from Marian received to be discussed at the next SECCOM.

China Mobile and Infosys would like to work on use cases. First call done yesterday, agreed on a model to move forward. Intent Based Mode would use Generative AI. 3 layers approach: business layer, services layer, domain layer. Each Intent Manager would have its own AI. Generic model would be used: business language into ONAP consumable, for services more data oriented and finally domain oriented. We do not focus on 5G only architecture but rather on any so could be used by any organization. 

Topic is in forming group. China Mobile and Infosys interested in Intent context. China Telecom is also interested with focused on user input and Intent.

Muddasar Ahmed Byung-Woo Jun Maggie update

China Telecom: New Delhi - data service. CCVPN use cases - LLM does not give enough intelligence. Develop domain specific model to generate more intelligent decisions.

China Mobile & Infosys: Intent based networking - Level 3 autonomy. Infosys is consulting with MNOs and has experience developing small AI-based autonomous loops.  New Delhi release: CM/Infosys will deliver LLM for Intent based networking. Intelligent decision making.

Post New Delhi will evaluate if the two tracks can leverage each other.

UUI is impacted system for both tracks. No impact to other components

NSA/Georgia Tech: AI/ML for security. Collecting and tagging security data to correlate the data.

Amy Zwarico provided reference to NIST AI 100-2e2023 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

Call this week booked. AI/ML use cases focus group still works on platform and use cases priorities.

CCVPN use case and intent based networking. China Mobile and Infosys starting work on that in more downstream.

ORAN WG11 is working on security aspects. WG2 (non-real-time RIC) and WG3 (near-real-time RIC) are working on xApps and rApps (AI/ML capabilities).

Need to write LF informative position white paper for AI/ML - team to write constituted. Meeting is planned with convenient time for all contributors. Goal is to produce it by DTF.

Structure bulleted paper available on Confluence - https://wiki.lfnetworking.org/pages/viewpage.action?pageId=120652848

China Mobile focus: generative AI. (New Delhi UUI)

China Telecom focus: intent transformation & LLM tuning parameters to create domain specific solutions. (New Delhi UUI)

Both projects are in progress.

Oslo lightweight model 

China Telecom and China Mobile presented at the last TSC their plans for AI/ML use cases with ONAP.

• China Mobile: "Intent Driven Orchestration for Autonomous Networks Leveraging GenAI, Nephio & ONAP" 
• China Telecom: "Exploration on Orchestration and Management of 6G mobile networks based on ONAP architecture"

By beginning of August both companies plan to present their plans for Oslo in this domain.

LFN AI-ML Use case formulation.pptx

Byung-Woo Jun informed SECCOM that the Nephio security WG is holding a joint meeting with the LF security SIG today at 11AM ET. Nephio plans to adopt 80% of OSSF passing badge.

Topic further discussed:

It was noted that the passing badge should be straight-forward to achieve.

The web page tlhansen.us/badging was discussed. Click on “Single Project…” then fill in a search string or badging ID (e.g. "nephio" or "7665").

For Nephio, Tony recommends to sort by “Type+Section”

Nephio SIG Security meeting:

By: Lucy Hyde When: Tuesday, October 31st, 2023 8:00am to 9:00am (UTC-07:00) Pacific Time - Los Angeles Repeats: Weekly on Tuesday Location: https://zoom.us/j/96025994457

We could support Nephio by sharing our best practices and processes in place. Lucy OOO for the next few weeks?

Byung introduced Tony's tool and was positively perceived by Nephio team. Nephio has GUI and talked about UI: AuthN and AuthZ to be shared by Byung.

Nephio Sig meeting last week: https://nephio.slack.com/files/U0503L9UA8N/F065V0AAZRQ/sig-security_action_items.pdf?origin_team=T03LMAUL4HH&origin_channel=D065DKWJJ9X 

No update - info collection ongoing. Byung will join SIG group. Secrets and Service Mesh

• Byung-Woo Jun many above items are done. LF Security and SIG Security joint meeting did not happen.

Nephio SIG Security discussion topics are:

• Secrets management leveraging Vault (open-source version)
• Service Mesh
• Ericsson plans to propose Identity and Access Management at the SIG Security meeting today (Jan 23)

Byung-Woo Jun Discussing R3 release.

• using open source vault for secrets storage
• Service mesh: ONAP uses a single management cluster. Nephio has a built-in service mesh component that can be added by the operator. E/// will propose IAM to SIG today.
• Considering OpenSSF tool.
• Muddasar Ahmed will provide a template for analysis.

Byung-Woo Jun The following proposals are under review at Nephio SIG Security

Nephio Secrets management user story proposal, https://docs.google.com/document/d/1Ce_cR7afovjWsdECkV8kNbPreG5GirfJXP5IrSiABjg/edit?usp=sharing

Service Mesh Requirements, https://docs.google.com/document/d/1UtW20GLTbICTUQyeC1Kx6aDnHlf4EqdhmeD29vsHSEM/edit?usp=sharing

Identity and Access Management Requirements proposal, https://docs.google.com/document/d/1qxGZI-HwTA0DfUO_hXKlkEpFzTNcmbDd6IO-CO7mLYo/edit?usp=sharing 

Package validation user story proposal, https://docs.google.com/document/d/1YeyUZUPFCS4bBgh8ShWVPrGs9HMLtrhwFSIDC6Xl3xc/edit?usp=sharing

Package validation under preparation.

Rahul Jadhav shared his Nephio workload identity. The team plans to review it. Also, Ericsson plans to share Identity and Access management requirements (2nd review) next week. requirements from Workload Identity perspective. 

https://docs.google.com/presentation/d/1K0gooS9ge181zNXLvA_SNAtGJyK1l77zOsRy6qpv7ME/edit?usp=sharing

Additional meeting planned today, E/// will provide user access control. Interest on workload to wokload access control, https://docs.google.com/document/d/1IwWVGASgdOuLHCHYg82WaZaHdOEXyOM1/edit#heading=h.30j0zll

Byung presented E/// user access control and workload access control under interest. SPIFFE in ORAN as study item. Workload identity still to be addressed.

Last Tuesday, Shiv Bhagavatula (Nephio SIG Architecture) shared additional Workload Identity design, leveraging SPIFFE infrastructure (SPIRE server, SPIRE agent, SPIFFE Id and SVID…), https://docs.google.com/presentation/d/1L79WrZ64Uar3IrH-jL_IeQTlPoLtXGZKHIIfVCXLoco/edit#slide=id.p.

Byung-Woo Jun , the latest user Identity and Access Management requirements, https://docs.google.com/document/d/1IwWVGASgdOuLHCHYg82WaZaHdOEXyOM1/edit#heading=h.nzahaii2p80p 

For Shiv and team’s workload identity design, https://docs.google.com/presentation/d/1L79WrZ64Uar3IrH-jL_IeQTlPoLtXGZKHIIfVCXLoco/edit#slide=id.g2bfc4581413_1_5

4/2: Target R4 (3Q24/4Q24)

Nephio Security team POC - (1) OIDC AuthN/AuthZ using Service Mesh and Key Cloak, (2) Workload identity and access management using SPIFFE.

Workload  identity and access management in progress. Internal discussions in E/// for next steps for user identity and access management.

• Workload Identity User Story, https://docs.google.com/document/d/1nkh7tTItwii1bY877PfzjFCBtmRos4IDh5EOJxWXRdg/edit#heading=h.srlm9p7wuua8
• Shiv plans to provide a better definition of "token".

Nephio R3, there is an action point, secret management, "sharing secrets across clusters as Skupper generates a secret in one cluster/namespace and that secret has to be shared with another cluster/namespace. To create the tunnel for communication. Now the question is how to share the secrets" It is a narrowed scope of sharing secretes between particular services, which is different from what Nephio SIG Security proposed.

Byung-Woo Jun , The Nephio SIG Security team (Shiv from Accuknox) plans to provide a demo of workload identity with SPIFFE this or next week. I will share the detail after their demo.

LF IT support is needed for SBOM SPDX format generation. Jess has experience with java based projects, and Nephio is Go based.

Byung-Woo Jun , Nephio O-RAN Workload Identity proposal by Nephio SIG Security will be presented to Nephio WG 2 ORAN on May 29th (postponed to June 5the), https://docs.google.com/presentation/d/1kofOHWswM2_OJPfefTcSzVvsBAg0QE3Z7GQITlaPO2w/edit#slide=id.p

Nephio Workload Identity execution plan:

• Start with PoC / demo to the relevant groups
• Requirements / user stories to SIG-1
• Detailed demo / run-through to SIG-Automation

Nephio update 2024-5-28:

• Signed image handling thru Nephio CI
• Nephio SGI security team is working on the above execution plan
• Nephio O-RAN workload identity proposal to Nephio WG 2 ORAN this week
• ORAN integration discussion (Q&A) further this week

Update xpected on 18th of June - Nephio signed image is a work in progress

Branch selected for Workload Identity - WIP.

• 2nd Demo for Workload Identity automation expected this week. 
• ORAN ZTA pdf: https://mediastorage.o-ran.org/white-papers/O-RAN.WG11.ZTA%20for%20Secure%20O-RAN%20White%20Paper-2024-05.pdf

Links shared by Muddasar:

https://datatracker.ietf.org/doc/bofreq-richer-wimse/

https://www.ietf.org/archive/id/draft-gilman-wimse-use-cases-00.html

Small demo presented with Vault used for secrets.

https://istio.io/latest/docs/ops/integrations/spire/

The second demo topic on July 23rd, 2024:

• Spire Server Agent created as part of the Nephio Install
• Node attestation for nodes in other clusters
• Node attestation for regional/edge clusters initiated by workload identity reconciler
• Library getting SVID, authenticating to vault, reading and writing to Vault
• Secrets CSI controller

Another demo on July 30th, 2024:

• SPIRE agent installed on all nodes via Damonset - how is registration of new nodes handled at scale?
• SA token is applied to SPIRE agent
• Spire-bundle is contained in a configmap for the spire agents to talk to the spire server
• How to prevent spoofing the SPIFFE ID/SVID? The current mitigation is JWT is 5 mins

Byung-Woo Jun TATA Communications supports RBAC, observability, logging, backup, etc. by leveraging the ONAP currently security mechanism (Ingress, Service Mesh) for their own platform. It is possible they contribute the enhancements they made to the ONAP New Delhi release (TBD).

Share of code most probably in Oslo release. Andreas is working on enhancements for OOM Team.

Tata communication shared which components in Montreal use STDOUT or not, ONAP Logging alignment for Montreal release.xlsx

Postponed to Oslo.

Ambient mesh under consideration if stable and memory safe.

Project Status in Oslo Release

Multicloud is currently not active, no PTL, no resources. Discussion with Andreas planned.

FY24 priority, security was covered - consensus on ONAP best practices.

http://tlhansen.us/badging

Platform Maturity Requirements (aka Carrier Grade)

New project induction and project graduation criteria documentation accepted. Security - discussion should be a separate WG meeting - security scrum of scrums. LFN Security Forum.

Updated meeting agenda for tomorrow's TAC meeting (https://wiki.lfnetworking.org/display/LN/2023-12-06+TAC+Minutes) and presentation planned by Amy and Muddasar:

• security scrum of scrums proposal
• Tony's dashboard for all LFN projects
• SAST and SCA tools and onboarding provided by LFN
• LFN having responsibility in releasing certifications (incubation, mature etc.)

TAC agreed with the proposal provided by Amy. In 6 months trial period we should have recommendations for secure software development. Projects SECCOM representatives to join those meetings. Sense of ownership to be improved.

LFN wide security focus group approved by TAC.

Align AI/ML initiatives

Creating LFN-wide Security FG

L3AF project - Microsoft pulled out

XGVela - no active contributors

FIDO

Muddasar Ahmed requested TAC to make a formal quality statement about LFN produced code.

CNCF certification and testing topic recently discussed.

Tailor from CNCF and Sana presented the need for certification. CNTI (CloudNative Network Function for Telco) conformance discussion - proposal expected by April to Governing Board.

Security whitepaper update under consideration - quality goal statement to be drafted. ORAN Alliance is doing yearly publication on security blog post.

CNTI - discussion finished last week. CNTI assets (test and documentation for certifying) moved to LFN.

Discussion on Superblueprint and documentation.

Migration process in progress.

Documentation update - modifying Lifecycle.

Confidential computing.

Leave automation, CI/CD security, sample statements under prep by Muddasar.

Having Superblueprint form as TSC. Aiming demonstration of projects interoperability. 

in San Jose joint LF Edge and LFN joint meeting. Muddasar to follow-up with LJ Luis.

CNTI still some discussion on migration plan for other conformance and certification efforts. Project is being approved, and some migration issues communication on certification path.

Quality goal could be more explored.

Meeting on Wednesday June 12th - CNTI induction discussion followed by induction of Paraglider candidate project - LF helps technology group coming to LFN. 5G Superblueprint considered in the future (matter of few months). 

CNTI approved as  project. Paraglide might prepare presentation. - Next meeting on July 10th.

KPIs in project promotion and with health check discussion with every project. Chair and vice Chair election. Security seat and Superblueprint seat.

Proposal: use Tony's 5 year assessment as a baseline.

Planning meeting done last week.

Muddasar Ahmed to check for document availability on software quality goals.

Muddasar Ahmed to follow with Jill.

Andreas was informed about lack of Jenkins jobs for Nexus-IQ scans. Fiete will work on this as project PTL.

Update from Fiete Ostkamp :

• maven based jobs use the lf-infra-sonatype-clm Jenkins builder that triggers the com.sonatype.clm:clm-maven-plugin
• for gradle there currently is no such builder available in global-jjb
• Sonatype officially does not provide an equivalent plugin, but there is the Sonatype Scan Gradle Plugin that could provide similar functionality

Jira opened by Fiete, ongoing support by LF-IT. Fiete is back from holidays.

Update from Fiete:

onap-portal-ng-preferences:

https://nexus-iq.wl.linuxfoundation.org/assets/index.html#/applicationReport/onap-portal-ng-preferences/b50d4e842a0847bc91437d354075e383/policy

onap-portal-ng-history:

https://nexus-iq.wl.linuxfoundation.org/assets/index.html#/applicationReport/onap-portal-ng-history/03f1fde4f7ea4f029031bbaf9689cfa8/policy

onap-portal-ng-bff:

https://nexus-iq.wl.linuxfoundation.org/assets/index.html#/applicationReport/onap-portal-ng-bff/4d9a28df94eb4bad85d858fe72321dac/policy