Showing posts with label Best Practices. Show all posts
Showing posts with label Best Practices. Show all posts

Monday, February 5, 2024

Overcoming the Barriers to Remote Work

Despite its many benefits, remote work also brings its own challenges. Understand what they are, and how to fix them.
Photo by Susanna Marsiglia on Unsplash

In previous articles, we discussed lots of best practices for remote and async teams. At this point, most understand the benefits of remote work for companies and employees. For companies, some of the benefits are global availability of resources, lower costs and increased capacity. For employees, seen are flexibility, global collaboration, increased autonomy, work-life balance, and more.

However, despite its popularity, remote work has its own challenges. So are some guidelines that you (and your team) should be aware of when working on a remote/async setting.

Barriers to Async work (and how to fix them)

Communication Gaps

Without real-time communication, there may be delays in responding to questions or issues, which can slow down decision-making and problem-solving. Make sure that documentation is always kept up to date and teams understand where to find it.

Misunderstandings

Written communication can sometimes lack the nuances of verbal communication, leading to misunderstandings or misinterpretations of messages. Establish a code of conduct for the team and foster safe zones of communication so that noise is minimized.

Time Zone Differences

When team members are spread across different time zones, coordinating work and communication can be challenging and it may lead to extended response times and difficulties in scheduling meetings or discussions. Document the different timezones your team operates in, and coach team members to respect their own personal time.

Overload of Information

Async work often involves written documentation and messages. Information overload can occur when there is too much documentation or communication to process effectively. Create effective documentation so everything is searchable, reducing the burden on one's memory.

Lack of Spontaneity

Async work may lack the spontaneity of face-to-face or real-time interactions, making it challenging to address urgent issues promptly. Make sure virtual happy hours exist with the purpose of social integration. If team members live near each other, facilitate in-person events.

Dependency on Written Skills

Some team members may struggle with written communication skills, leading to less effective collaboration in an async environment. If one can't write good documentation, distribute the task among other team members.

Difficulty in Fostering Creativity

Collaborative brainstorming and idea generation may be less effective in async settings where spontaneous discussions are limited. Leverage virtual boards such as Miro (or Figma) and encourage everyone to collaborate.

Dependency on Technology

Async work relies heavily on technology and digital tools. Technical issues or downtime can disrupt work and communication. Make sure team members go through necessary trainings, that they understand how to effectively communication and have proper access to documentation. Finally, encourage recordings so the unavailable can follow up later.

Accountability

Ensuring that tasks are completed and deadlines are met can be more challenging in an async environment without real-time monitoring and follow-up. Make sure teams understand the project deadlines. Use boards like Jira that increase accountability and awareness of the deliverables.

Cultural Differences

Different cultures have varying norms and expectations around communication and work styles. These differences can lead to misunderstandings and conflicts in async teams. Make sure the environment is respectful, and team members understand and respect cultural differences.

Conclusion

To overcome these barriers, organizations and teams can implement strategies and best practices, such as providing training, setting clear expectations, establishing communication protocols, and selecting appropriate tools.

However, it's essential to recognize that async work is not suitable for all situations and to strike a balance between async and synchronous work when necessary to meet the needs of the team and the nature of the work being done.

Tuesday, January 2, 2024

Dialogflow enters the GenAI / LLM era

Building a chatbot or voice chatbot? Check how the latest AI technologies added to Dialogflow could be the answer for your next project
Photo by Google DeepMind on Unsplash

In the Google Cloud Next '23Google announced many AI enhancements to Dialogflow, including addition of GenAI and LLM capabilities. Because these are groundbreaking changes to the very popular Dialogflow, wanted to explore howthe new GenAI features added to Dialogflow CX transform it in a fantastic service for a conversational chatbot and voice-chatbot.

Why (re)consider Dialogflow

Those who worked with Dialogflow in the past years, often say that a conversational chatbot built with Dialogflow gets complicated pretty quickly given the many variations a business should consider to implement a useful chatbot or voice chatbot.

This may have changed since Google Cloud Next '23, as Google announced many AI enhancements to Dialogflow, including addition of genai and LLM capabilities.

In this article we review how good the new GenAI features are, and how they take Dialogflow to the next level.

To test those features and understand how they augment Dialogflow, we built a real voice chatbot (plugged on a real telephone number). Overall, the results were fantastic! The latest GenAI features augment Dialogflow in a way that building a voice chatbot is quick, fun, pleasant and intuitive.

So let’s understand what is it that Google brought to the platform.

New GenAI features

If we had conducted this spike in Summer 2023, we probably wouldn’t be able to recommend Dialogflow CX for an intelligent voice/chatbot. Why? Because without the GenAI features announced in August 2023, it would not be possible to build an organic conversation with Dialogflow, without a lot of customization.

But how quickly this changed!

With the latest GenAI features announced for Dialogflow, building a voice chatbot is quick, fun, pleasant and intuitive.

So we went to explore the solution with some scepticism, and to our surprise, the results were great! Uploading custom data to the bot, having it trained and building a conversation on top of it was fast and intuitive.

In summary, here are some of the most interesting features announced in Fall 2023:

  • Ability to tailor your ML models
  • Ability to upload your own documents
  • Ability to fine-tuned your ML models
  • Ability to customize the responses with easy upload of documents. This could be great to simply upload internal/specific docs on/about the company, and have the model index that,
  • Leverage hybrid agents (combines the power of precise conversation controls (flows, parameters, intents, conditions, transitions, and so on) with data store handler generative features), as presented in our demo.
  • Ability to handle undesired intent matches
  • Ability to store internal documentation

Google Cloud GenAI Architecture

So let's review what's changed.

To make the Dialogflox CX + genai experience great, Google has created an impressive layer of optimization (depicted on the image above), which makes any custom made tough to beat.

Source: Google Cloud Next '23

Our Implementation

To validate the new features we wanted to build a voice chatbot using custom data, with little customization on Dialogflow's part. Was it be possible?

Indeed! We managed to deploy a fully operational bot in only 4 hours of work! Here is what we achieved during this time:

  • Uploaded custom docs
  • Fad the model auto-trained to answer questions on those docs (screenshots below)
  • Created custom intents to guide the conversation
  • Integrated with telephony providers, called it over the phone, and it was able to respond to random questions elegantly
  • Tested and tuned the solution
So let's get into it the most interesting features of the platform.

Features

Side by side Apps

Business would be able to have side by side apps. Below, we show two GenAI apps built for different clients. This would allow clients to have multiple apps, tailored to different business uses, for example.


Data Stores and custom training

Different solutions also have their own custom data store. Data stores are used to find answers for end-user's questions, and can be sourced from websites and/or custom documents. Below we show two possible options: website indexing (crawling) and document upload.

Once the data is ingested, the model is automatically trained on it leveraging automated or advanced NLU capabilities:


Asking custom questions

As your bot learns from your data, it becomes able to answer questions based on the documentation provided. Note that this required no training from our part.

Our chatbot responding to questions related from an uploaded doc, and from custom training.

Testing your bot

On the image above, you can also see a textbox ("talk to an agent"), where you can interact with the bot via text. This makes sense as we don't want to be test our bot over the phone, if we can do it from Dialogflow's UI, right?

Custom data importing

As mentioned, custom data can be easily imported using the admin portal. Here's what it looks like.

Auditing

Changes and modifications to data are also tracked for auditing and compliance reasons. This is extremely important for business, and a feature that should not be ignored.

Inline Testing Simulator

The interface also allows us to quickly test (and even listen) to the model, as shown below.

Test Cases

To test your agent, you create test cases using the simulator to define golden conversations, then you execute test cases as needed. This article explains that in more detail. You can also use the built-in test feature to uncover bugs and prevent regressions.

Custom Training

Beyond data stores, it's also possible to provide custom training to match to specific intents. For example, here we train the bot to respond specific questions the client might ask.


Response Tuning

Of course, the response can be fine tuned for any specific intent. The image below shows a custom messaged we trained the bot with, so it could respond elegantly to the calling client, looking to pay their outstanding invoice.


Backend integration

Integration with the backend is achieved via fulfilments, a Dialogflow term. Integrations with external backends can be configured on the Webhooks admin section. Those webohooks are leveraged by the fullfilment API, directly by Dialogflow.


Generators

Generators allow you to use generative AI models to generate dynamic responses or text that can be used during fulfillment. The next image shows some of the tuning you can do with your model.

Custom Voice

Of course, any serious solution for a voice chatbot should allow customizing the voice of your bot,. With Dialogflow that's not different. You can also leverage mult-language, and even create a new bot voice based on any human voice.


Advanced speech settings

Beyond the features mentioned above, it’s also possible to tune the speech in great details. Below you can see the different options at your disposal.

Multi-language

Support for multi-language (and locales) is also available. This is great for organizations that operate on more than one language, as it's common here in North America.


Security

Security is an essential requirement for every implementation. Dialogflow provides a robust set of security features and a great level of customization. Security Settings represents the settings related to security issues, such as data redaction and data retention. It may take hours for updates on the settings to propagate to all the related components and take effect. Learn more



AI, GenAI and LLM

So let's review some features that leverage AI directly.

Agent Settings

The Agent Settings screen allow to fine tune your agent with specific settings.

GenAI Settings

The Generative AI settings also provides a specific section for GenAI tuning.

Data Store Prompt

Data store prompt is a type of prompt engineering, where you can fine tune the answers given by your model. Dialogflow addresses this very elegantly by making it extremely simple to customize your inputs via a Data store prompt setting.

Sentiment analysis

Sentiment analysis inspects end-user input and identifies the prevailing subjective opinion, especially to determine an end-user's attitude as positive, negative, or neutral. When making a detect intent request with the API, you can specify that sentiment analysis be performed, and the response will contain sentiment analysis values. Learn more

Generative Fallback

The generative fallback feature uses Google's latest generative large language models (LLMs) to generate virtual agent responses when end-user input does not match an intent or parameter for form filling. The feature can be configured with a text prompt that instructs the LLM how to respond. You can use a predefined text prompt or add your own prompts.

Conversation Playbook

Vertex AI Conversation’s playbook feature (in preview) lets you use natural language to define what responses and transactions you want to enable your voice and chatbots to perform, similar to how you would instruct a human agent on how to handle tasks.


Answer feedback

Finally, answer feedback helps track agent performance, Dialogflow provides tools for collecting and analyzing end-user feedback on agent answers during a conversation.

Our Assessment

As you can see, the new GenAI capabilities of Dialogflow are impressive! It's possible now build and deploy to production, a genai based voice chatbot in less the 4 hours!

In summary, here is why you should consider Dialogflow for future chatbots, or voice-powered chatbots:
  • Robust ecosystem of features
  • Robust out of the box integrations (seamless integration to telephony, seamless integration with other services (SMS, text, chat) including WhatsApp, Facebook Messenger, Teams and Slack)
  • Robust console to manage flows and configure the experience
  • Native GenAI capabilities (as of Aug 29th 2023).
  • Ability to customize the responses with easy upload of documents. This could be great to tailor the answers with company specific content.
  • Custom workflows (also known as fulfilment) via webhooks.
  • Extremely high ROI (our team managed to have an operational voice chatbot in just 4 hours of work).
  • Voice customization (you can even train the bot to use your voice)
  • Multi-language
  • Ability to leverage hybrid agents combining the power of precise conversation controls (flows, parameters, intents, conditions, transitions, and so on) with data store handler generative features), as presented in our demo.
  • Ability to handle undesired intent matches (including backlisting of answers and topics)
  • Ability to have separate environments
  • Ability to test the models inline
  • NLP, NLU, TTS, STT and other AI features provided out of the box
  • Ability to conduct experiments
  • Ability to integrate with CI/CD pipelines (see screenshots below)
  • Analytics on calls and conversations
  • Ability to create experiments (aka. A/B/n testing)
  • Extensive API to build voice chatbots directly from code
  • Enterprise-grade GRC (governance, risk and compliance)
  • Fully managed solution

Conclusion

The ecosystem for AutoML, LLMs and GenAI is thriving with new solutions being created every day. Google is investing heavily in genai to outsmart competition, and is adding AI to most of its enterprise offerings. As an example, the features listed above are extremely new (announced last week), with others still to be added.

With that said, Dialogflow CX became a great alternative for voice chatbots if the advantages listed above can be justified over its tradeoffs: a more opinionated solution, less room for customization, requirement for Google Cloud, and more (yet not clear).

In counterpart, a custom solution could give us much more opportunity for customization to adapt to client’s needs. However, it too would have its own tradeoffs: a slower pace of development, higher cost of implementation, non-managed ops, a moving target on security, increased complexity, and more (yet not clear).

Monday, December 18, 2023

Running effective asynchronous (async) meetings

In the remote/asynchronous world we live in, running effective asynchronous (async) meetings is becoming a popular reality. Here's what you need to know to succeed.
Photo by Christin Hume on Unsplash

Hey, good news! In the remote era, running effective asynchronous (async) meetings is possible! On this post, let's understand how to succeed, run and participate in async meetings.

What is an async meeting?

What do you understand by "async meeting"? Yes, it's essentially a meeting that happens asynchronously, where participants engage without the need for same-time communication. Its biggest benefit is asynchronous collaboration, especially in remote or distributed work environments where team members work in different locations of the globe.

Best Practices for Async Meetings

So let's review some tips on how to run effective (and productive) async meetings.

Set Clear Objectives

First, you should define the purpose and goals of the meeting. What specific outcomes or decisions does the team want to achieve?

Choose the Right Tools

Next, make sure you utilize collaboration and communication tools designed for async meetings. Popular options include Slack, Microsoft Teams, Google Meet, or dedicated async meeting platforms.

TIP: Check our previous post on async work for more information.

Schedule in Advance

Announce the async meeting well in advance to allow participants to plan their time and contribute thoughtfully.

Provide Context

Share background information, documents, and relevant context before the meeting. Make sure participants have all the necessary information to contribute effectively.

Set a Deadline

Because async meetings don't end at the same time for everyone, it's important to specify a deadline for participants to provide their input or feedback. This creates a sense of urgency and helps keep the meeting on schedule.

Agenda and Structure

Create a clear agenda that outlines the topics, questions, or tasks to be addressed during the meeting. Organize the meeting into structured sections to make it easy for participants to follow and respond to specific points.

Use Clear Communication

Write clear and concise messages or prompts. Avoid overly lengthy or ambiguous messages that could lead to misunderstandings.

Encourage Participation

Before, during and after the event, make sure that everyone understands that their input is valuable. Encourage participants to contribute their ideas, suggestions, or feedback openly.

Set Expectations

As sync meetings, async meetings have a goal. Clearly communicate the expected level of participation and the timeline for responses. Be explicit about the desired outcome for each participant, or for the team.

Follow-Up and Reminders

Another good practice is to send reminders and follow-up messages to participants as the deadline approaches. Ensure that no one forgets to contribute.

Consolidate Responses

Once all responses are collected, consolidate and summarize them into a cohesive document or message. Highlight key points, decisions, and action items.

TIP: Use GenAI to summarize the information. Why not?

Document the Meeting

Keep a record of the meeting's outcomes, decisions, and action items. Share this documentation with all participants for reference.

Decision-Making Process

If the async meeting is intended for decision-making, clearly outline the decision-making process and criteria. Use a voting system or other method to reach a consensus.

Review and Iterate

After the async meeting, evaluate its effectiveness. Gather feedback from participants and make improvements for future async meetings.

Considerations

But async meetings has its challenges too. Here are some details that organizations should consider.

Manage Overload

Be cautious about overloading participants with too many async meetings. Prioritize and schedule them judiciously to avoid burnout.

Assign Roles

Designate a facilitator or organizer responsible for managing the meeting and ensuring that participants stay on track.

Accessibility and Inclusivity

Ensure that all participants have equal access to the meeting materials and can contribute comfortably. Consider time zones and different work schedules to accommodate global teams.

Respect Time Zones

Be mindful of time zone differences when setting deadlines and sending reminders. Use tools that display multiple time zones to avoid confusion.

Conclusion

Effective async meetings can enhance collaboration, allow for thoughtful contributions, and accommodate the diverse schedules and preferences of team members in remote or distributed work settings. It's important to establish clear processes and expectations to make async meetings a valuable and efficient part of your team's workflow.

Friday, December 1, 2023

Creating Great Documentation for Remote Teams

Remote work is not only about writing code and creating PRs, but also making sure key information can be easily found

Continuing on the topic remote work and async best practices, today we will cover how to leverage documentation to effectively. 

Creating great documentation for use in remote teams is essential for ensuring that team members have access to the information and resources they need to work effectively, even when they are not physically located in the same place.

Why documentation is important

Documentation is a foundational concept for IT and software development teams. Some of the obvious benefits are: greater collaboration, knowledge sharing, problem-solving, and quality assurance while also serving as a reference for future development and maintenance efforts.

Documentation plays a crucial role in the context of information technology (IT) and software development for several reasons, as it can be used for:

  • Knowledge Transfer
  • Onboarding
  • Maintenance and Troubleshooting
  • Compliance and Auditing
  • Quality Assurance
  • Risk Mitigation
  • Collaboration
  • Scalability
  • Stakeholder Communication
  • Historical Context, and more.

Given its critical importance, let's take a look at some best practices for effective documentation commonly used by remote teams.

Documentation Best Practices for Remote Teams

Define Clear Objectives

Clearly state the purpose and objectives of the documentation. What specific information or tasks should it help remote team members accomplish?

Structure and Organization

Organize the documentation logically with a clear structure. Use headings, subheadings, and a table of contents to make it easy to navigate. Group related topics together, and use a consistent naming convention for files and sections.

Don't forget with the Basics

Begin by documenting fundamental information such as team goals, mission, vision, and core values. Provide an overview of the team's structure and members.

Include Detailed Guides

Develop detailed guides, tutorials, and how-to documents for common tasks and processes. These guides should be step-by-step and include screenshots or videos if necessary. Consider creating templates for common documents, reports, or forms to ensure consistency.

Use Visuals and Examples

Incorporate visuals, diagrams, and examples to enhance understanding. Visual aids can be particularly helpful for complex concepts or processes.

Keep It Updated

Regularly review and update the documentation to ensure its accuracy. Assign ownership of specific sections to team members responsible for the content.

Incorporate a Feedback Mechanism

Establish a feedback mechanism for team members to suggest improvements or report errors in the documentation. Act on this feedback promptly.

Facilitate Regular Trainings

Provide training to remote team members on how to use the documentation effectively. Make sure they understand how to access and navigate the documentation platform.

Don't forget Onboarding Material

Create a dedicated section for onboarding materials for new remote team members. Include information on team culture, policies, and procedures.

Promote Usage

Encourage remote team members to use the documentation as their primary source of information and guidance. Emphasize its importance in remote work.

Identify Your Audience

Determine who will be using the documentation. Remote teams often consist of a diverse group of individuals with different roles and responsibilities.

Essential Tools for Effective Project Documentation

Without supporting tools, great documentation would not be accessed effectively. So let's review essential tools to maximize your team's use and access to documentation.

Version Control

Implement version control for documentation so that team members can access previous versions if needed. This is crucial for tracking changes and maintaining a historical record.

Search and Indexing

Implement a robust search function or indexing system within your documentation platform to help team members quickly find the information they need.

Foster Collaborative Editing

Always allow team members to contribute to and edit documentation collaboratively. Encourage subject matter experts to share their knowledge.

Review Access Control

Manage access to documentation carefully. Ensure that team members have appropriate permissions to view and edit documents based on their roles and responsibilities.

Backup and Disaster Recovery

Implement a backup and disaster recovery plan to ensure that documentation remains accessible even in the event of technical issues or data loss.

Security and Privacy

Consider security and privacy when creating and sharing documentation, especially when dealing with sensitive information. Use encryption and access controls as needed. Conduct regular audits or reviews of the documentation to identify areas for improvement or expansion based on changing needs and processes.

Which tools should I use?

First and foremost, use what is allowed by your organization,  and what works for your team. Consider using collaborative tools like Google Docs, Microsoft SharePoint, Confluence, or dedicated documentation software like Notion or Markdown-based systems like GitHub.

Just don't forget to tune your security, and get everyone onboarded to access the information they need.

Conclusion

Great documentation is an essential resource for remote teams. It fosters clarity, consistency, and productivity, enabling team members to work effectively and independently regardless of their physical locations. Regularly maintaining and improving documentation is an ongoing process that contributes to the success of remote work environments.


Wednesday, November 15, 2023

How to work effectively in an asynchronous world

More and more people work remotely. Let's review essential best practices for you (and your teams) to succeed.
Photo by Avi Richards on Unsplash

Working effectively asynchronously, especially in remote or distributed teams, requires careful planning, communication, and time management. Asynchronous work means that team members are not necessarily working at the same time or in the same location.

Recommended Best Practices

When working remotely (and asynchronously), there are many details that you (and your team) should pay attention to, not to sacrifice the end goals of your project. So let's review some strategies for working effectively in an asynchronous environment.

Clear Communication

Make sure everyone in the team can communicate effectively. To be a successful async employee, clearly articulate your thoughts, expectations, and questions in emails, messages, or collaboration tools. Be concise and organized in your written communication to ensure your message is easily understood.

Use Collaboration Tools

Utilize project management and collaboration tools like Slack, Microsoft Teams, or Asana to centralize communication and track tasks. Use shared documents and cloud-based platforms (e.g., Google Workspace, Microsoft Office 365) to collaborate on files in a distributed setting.

Set Expectations

Establish clear expectations for your response times. Make it known when a response is needed and when it's acceptable to respond. Document guidelines for working asynchronously within your team, including preferred communication channels and response windows.

Structured Workflow

Define processes and workflows that allow team members to work independently and efficiently. This might include creating templates or checklists. Set deadlines and milestones to keep projects on track and ensure everyone knows their responsibilities.

Time Management

Plan your day and allocate time for specific tasks. Prioritize high-impact, strategic work and minimize time spent on low-value activities. Use time management techniques like the Pomodoro Technique or time blocking to stay focused and productive.

Provide Regular Updates

Provide updates on your progress to keep team members informed. This can be in the form of daily or weekly status reports. Share your accomplishments, challenges, and next steps so that others can easily catch up on your work.

Effective Documentation

Document decisions, discussions, and important information in a central location. This ensures that team members can refer back to information when needed. Consider creating a knowledge base or wiki for your team or organization.

Consider Time Zones

Be mindful of time zone differences when scheduling meetings or setting deadlines. Use scheduling tools that can display multiple time zones. Avoid scheduling urgent meetings during non-standard working hours for team members in different time zones.

Make Information Available

Use status indicators in communication tools to let others know when you're available for quick questions or discussions. Set boundaries for when you're not available to prevent burnout.

Collaborative Decision-Making

When making important decisions, involve all relevant team members asynchronously by sharing information and gathering feedback through written channels.

Provide Feedback and Recognition

Provide feedback and recognition to your colleagues through written praise or constructive feedback to acknowledge their efforts and contributions.

Foster Continuous Learning

Embrace continuous learning and improvement. Stay updated on new tools and best practices for asynchronous work.

Leverage a Data-Driven Approach

Analyze data on team performance, response times, and project progress to identify areas for improvement in your asynchronous workflow.

Don't forget Team Building

Organize occasional synchronous meetings or team-building activities to maintain social connections and strengthen relationships.

Conclusion

Working effectively asynchronously requires discipline, adaptability, and strong communication skills. By implementing these strategies and continuously refining your approach, you can create a productive and collaborative work environment, even when team members are not working in the same time zone or location.

Wednesday, November 1, 2023

OWASP: Your Guide to Secure Web Development

With cyber threats on the rise, more and more developers are expected to build more trustworthy applications. Learn how
Photo by Ante Hamersmit on Unsplash

Cyber threat is a trend that unfortunately will not go away any time soon. In fact, it will just keep growing. As as developers, it's our duty to build solutions that are reliable and resistant to attacks, which is a complex undertaking.

Fortunately, the OWASP project provides a lot of information on how to secure applications. It's available for free, and is created and maintained by security experts. So let's learn more about it.

What is OWASP

For starters, OWASP is an acronym for:
  • O – Open
  • W – Web 
  • A – Application
  • S – Security
  • P – Project

The Open Web Application Security Project is an online community that produces freely-available articles, methodologies, documentation, tools, and technologies in the field of web application security.

OWASP is non-profit organization community of international security researchers and experts dedicated to improving the security of software, with a especial focus on AppSec (application security).

What it offers

Contrary to what you might think, OWASP is not only about documentation. Here are some highlights of what the project offers:

  • Wide range of resources to help organizations mitigate security threats and reduce their exposure.
  • Extensive documentation on cybersecurity practices
  • Tooling to learn, test and validate different aspects of security
  • Resources that help identify and mitigate security vulnerabilities in their web applications and APIs.
Some of my favourite resources are:
  • OWASP Top Ten (ten most popular threats in AppSec)
  • OWASP Projects (extense and diverse compilation of projects and tools, as we’ll see)
  • Extensive technical documentation
  • Chapters (community for application security professionals around the world)
  • Conferences
  • Web Application Security Testing Guidelines (WSTG)
  • Education and Training
  • Industry Reports

Let's learn more about them.

When, How and Why leverage OWASP

To keep it simple, you should use OWASP whenever you are building any application (client-facing or not) that interacts with data and is used by users (in other words, for most projects deployed in production).

More importantly, you should leverage OWASP because:

  • Security and AppSec are HARD!
  • Security is a moving target
  • It offers a collection of best practices from security experts
  • It is continuously updated to cover most popular attacks in AppSec
  • Btw, did I mention that security is HARD?

Don’t implement security related features “your way”. Most likely it is not secure enough. Leverage well established patterns such as those provided by OWASP.

Flagship Projects

So let's take a look at some flagship projects.

OWASP Top Ten

One of OWASP’s most popular projects, OWASP Top 10 is a reference standard for the most critical web application security risks. The community regularly updates the list with the ten most critical (and popular) web application security risks.

Active for over 20 years, the project receives contribution from the international community of security experts and researchers. One of its benefits is to bring awareness to the most critical attacks, as well as helping developers and security professionals prioritize their efforts in securing web applications.

Here are is the top 10 attacks in 2021:

OWASP Top 10 2021. Source: OWASP

OWASP Cheat Sheet Series

Another essential resource for building secure web applications, the OWASP Cheat Sheet Series provides easily accessible practice guides for application developers and defenders to follow.

The project offers more than 80 cheat sheets and security best practice in form of guides for application developers and defenders to follow.

You should leverage it as it helps developers and security professionals prioritize their efforts in securing web application.

OWASP Dependency-Check

OWASP Dependency-Check is a Software Composition Analysis (SCA) tool suite that identifies project dependencies and checks if there are any known, publicly disclosed, vulnerabilities.

OWASP Juice Shop

OWASP Juice Shop is a very sophisticated (and insecure) web application for security trainings. Also great voluntary guinea pig for your security tools and DevSecOps pipelines!

Getting started with Juice Shop is easy! Check this GitHub page for more information.

OWASP Mobile Application Security

The OWASP Mobile Application Security project offers security standards for mobile apps and a comprehensive testing guide that covers the processes, techniques, and tools used during a mobile application security assessment.

OWASP Web Security Testing Guide

The OWASP Web Security Testing Guide project produces the premier cybersecurity testing resource for web application developers and security professionals. A PDF is available for free on from GitHub.

Some highlights of WSTG:

  • Fantastic guide to testing the security of web applications and web services.
  • Created by security professionals and dedicated volunteers
  • Framework of best practices used by penetration testers all over the world.
  • 450+ pages of AppSec!
Don't forget to download your WSTG PDF directly from GitHub.

OWASP ZAP

One of my favourite ones, OWASP ZAP is the world’s most widely used web app scanner. Free and open source. Actively maintained by a dedicated international team of volunteers. ZAP is a free alternative to the very popular (and excellent) Burp Suite.

Some of the features available on OWASP ZAP:

  • Automated Scanning
  • Manual Testing
  • Spidering and Crawling
  • Active and Passive Scanning
  • Alerts and Reporting
  • Session Management
  • Fuzzer
  • Authentication Support
  • Plug-in Support
  • WebSocket Testing
  • Automation and Integration
  • Community and Updates
  • Multi-Platform Support

OWASP Amass

The OWASP Amass tool Performs network mapping of attack surfaces and external asset discovery using open source information gathering and active reconnaissance techniques..

Conclusion

As cyber threats grow, developers should protect their applications from increasingly complex and sophisticated attacks. For that, OWASP is an essential project to know, study and use.

Hope it helps.

Tuesday, October 3, 2023

How to be a developer in the AI era

Contrary to predictions, most developers won't lose their jobs to ChatGPT. Here is why

Photo by Possessed Photography on Unsplash

The beginning of 2023 brought us a big buzz with the release of ChatGPT. Impressed with its capacity and that of LLM-based solutions (like Bard, GitHub Copilot and Google Duet AI), the community started questioning the role of developers, and if they'd soon be replaced by a coder robot.

On this post let's understand why that's not happening anytime soon, and what you need to do to survive the genai era.

Essential skills for any developer

To be a developer in the AI era, first and foremost you need strong CS skills. That means strong foundation in computer science fundamentals, systems design, databases, web applications, design patterns, and more.

With that in mind, here are essential skills you should work on to remain valuable in the marketplace.

Programming languages

Developers need to be able to write code in at least one programming language. Popular programming languages include Python, Java, JavaScript, Go, Rust or C#.

Technical Skills

For technical skills, include everything you need to do your work well. On this list you should include automation, frameworks, cloud, containerization, databases, infrastructure, security, storage, and more.

Data structures and algorithms

Developers need to have a good understanding of data structures and algorithms in order to write efficient and effective code. Make sure you understand well data structures and algorithms to build efficient solutions, compare the performance of different solutions, and more.

Problem-solving skills

Developers need to be able to identify and solve problems. This includes being able to break down complex problems into smaller, more manageable problems.

Critical thinking skills

Developers need to be able to think critically about their code and the problems they are trying to solve. This includes being able to identify and correct errors in their code.

Communication skills

Developers need to be able to communicate effectively with both technical and non-technical audiences. This includes being able to explain complex technical concepts in a clear and concise way.

Teamwork skills

Developers often work on teams to develop and maintain software. This means that they need to be able to work effectively with others, share ideas, and collaborate on projects.

Keep learning

On top of it all, make sure you understand the latest trends in your particular field (ex. web, mobile, cloud, etc).

Becoming a successful developer in the AI era

As you build the skills above, a parallel goal for you would be developing a good understanding of AI concepts and technologies. 

Here are some specific tips on how to become a successful developer in the AI era:

Learn about AI concepts and technologies

Make sure you understand fundamental AI concepts and technologies, including topics such as machine learning, deep learning, and natural language processing.

Start using AI tools

Use AI tools! That includes testing GitHub Copilot or other tools that optimize the way you work. There are so many of them, and the ecosystem is changing so rapidly that I recommend you exploring on your own.

Learn a programming language for AI development

Popular languages for AI development include Python, R, and Julia.

Build AI projects

The best way to learn AI is by doing. Try building your own AI projects, such as a chatbot, image classifier, or recommendation system. Make sure you understand them. How could they be used in your project?

Contribute to open source AI projects

Learn from other developers and gain experience working on real-world AI projects.

Stay up-to-date on the latest AI trends and developments

The field of AI is constantly evolving, so it is important to stay up-to-date on the latest trends and developments.

Growing your skills to the next level

Finally, you should never stop learning or growing your skills. With that in mind, here are some additional tips that may be helpful.

Become an expert in your chosen field

This will make you more competitive in the job market and make it easier to find high-paying jobs.

Develop your soft skills

Soft skills, such as communication, teamwork, and problem-solving, are essential for success in any field, but they are especially important in the AI era.

Network with other AI developers

This is a great way to learn new things, find job opportunities, and collaborate on projects.

Conclusion

The AI era is an exciting time to be a developer. There are many new opportunities for developers who have the skills and knowledge to build AI applications. By following the tips above, you can prepare yourself for a successful career as an AI developer.

Monday, September 18, 2023

Why Java still matters

With the release of Java 21, the conversation on the relevance of the venerable programming language restarted. But a more important question remains, does Java still matter?

Photo by Mike Kenneally on Unsplash

What announcement would you like to see in Java 21 that would make you excited? Well, for most people, actually nothing. Because, you know, what's exciting with a predictable, slow-paced, stable, widely adopted enterprise programming language?

Read to find out.

A little bit of Java History

Java was originally developed by James Gosling at Sun Microsystems in 1991. The goal was to develop a small, reliable, portable, distributed, real-time operating platform. The language was initially called Oak, later renamed to Green and finally renamed Java, from Java coffee, a type of coffee from Indonesia.

Officially released in 1995, Java quickly became popular for developing web applications. The Java Virtual Machine (JVM) allowed Java code to run on any platform that had a JVM, which made it a very versatile language. Java is also a very secure language, which made it a good choice for developing web applications.

Key events in Java's history

For brevity, here are some of the key events in the history of Java:

  • 1991: James Gosling starts the Java project at Sun Microsystems.
  • 1995: Java 1.0 is released.
  • 1996: Java becomes a popular language for developing web applications.
  • 1998: Java 1.1 is released, adding new features such as garbage collection and threads.
  • 2000: Java 1.2 is released, adding new features such as swing and applets.
  • 2004: Java 5 is released, adding new features such as generics and annotations.
  • 2009: Java 6 is released, adding new features such as concurrency improvements and a new security manager.
  • 2014: Java 8 is released, adding new features such as lambda expressions and streams.
  • 2017: Java 9 is released, adding new features such as modules and a new garbage collector.
  • 2018: Java 10 is released, adding new features such as a new date and time API.
  • 2019: Java 11 is released, adding new features such as a new HTTP client and a new text block literal.
  • 2020: Java 12 is released, adding new features such as switch expressions and sealed classes.
  • 2021: Java 13 is released, adding new features such as text blocks and pattern matching.
  • 2022: Java 14 is released, adding new features such as sealed interfaces and record classes.
  • 2023: Java 21 is released.
Looks like a pretty good cadence for a 30 year old programming language.

Applications

So where does Java run? Well, these days Java runs pretty much everywhere:

Enterprise Application Development

Java is commonly used to build large-scale, mission-critical applications for business operations. These applications can include customer relationship management (CRM) systems, enterprise resource planning (ERP) software, and supply chain management solutions.

Web Development

Java is used to build robust and scalable web applications. Companies often use Java-based frameworks like Spring and JavaServer Faces (JSF) to create web applications that handle high traffic loads, such as e-commerce websites and online banking platforms.

Mobile App Development

Java is one of the primary languages used for Android app development. Many Fortune 1000 companies develop Android applications for their customers, employees, or partners

Big Data and Analytics

Java is used in big data processing and analytics applications. Companies leverage Java libraries and frameworks, such as Apache Hadoop and Apache Spark, to analyze large volumes of data and gain insights for decision-making.

Middleware and Integration

Java is often used to develop middleware components and integration solutions. These components help connect various software systems and applications within an organization, enabling data flow and communication between them.

Cloud Services

Java is used to develop and run applications on cloud platforms like AWS, Azure and GCP. Java-based microservices and containers are common in cloud-native architectures.

Internet of Things (IoT)

Java can be used to develop IoT applications and solutions. It's used to create firmware for IoT devices and build backend systems that collect and process data from these devices.

Security

Java's security features are essential for companies. It's used in developing secure authentication systems, encryption algorithms, and secure communication protocols to protect sensitive data.

Financial Services

Many large financial institutions rely on Java for their trading platforms, risk management systems, and banking applications due to Java's performance and reliability.

Customer Support

Java is used to develop customer support systems, including chatbots and helpdesk applications, to improve customer service and streamline support operations.

Supply Chain and Logistics

Companies use Java to build applications that manage and optimize their supply chain, inventory, and logistics operations, helping them reduce costs and improve efficiency.

Content Management Systems (CMS)

Java-based CMS platforms are used for managing and delivering digital content on websites and other digital platforms.

E-commerce

Java is frequently used for developing e-commerce platforms, shopping carts, and payment processing systems to support online sales operations.

Data Warehousing

Java is used in the development of data warehousing solutions that enable companies to store, process, and retrieve large volumes of structured and unstructured data for analysis and reporting.

What's new in Java 21

Finally, let's jump into Java 21 (released in September 2023). The release adds many interesting features are being added to the language including:

  • Virtual Threads: Virtual threads are a new lightweight threading abstraction that can be used to improve the performance of multithreaded applications.
  • Record Patterns: Record patterns are a new feature that can be used to deconstruct record values in a more concise and readable way.
  • Pattern Matching for switch: Pattern matching for switch is a new feature that can be used to match values in a switch statement in a more concise and readable way.
  • Sequenced Collections: Sequenced collections are a new type of collection that provides direct access to the first and last elements of the collection.
  • String Templates: String templates are a new feature that can be used to simplify the process of string formatting and manipulation.
  • Unnamed Classes and Instance main() Methods: Unnamed classes and instance main() methods are preview features that can be used to simplify the code for small, self-contained classes and methods.

Does Java still matter?

But the question remains: does Java still matter? Well, given the extensive reach and adoption of the language, the answer is of course, a lot!

But in case you are still not convinced, here are some reasons why Java still matters:
  • Popularity: Java is still a very popular programming language today. It is used by millions of developers around the world and is the language of choice for many large and complex applications.
  • Portability: Java code can run on any platform that has a Java Virtual Machine (JVM). This makes it a very versatile language that can be used to develop applications for a wide range of devices.
  • Robustness: Java is a very robust language that is designed to be secure and reliable. It has a number of features that help to prevent errors and crashes, such as garbage collection and exception handling.
  • Performance: Java code can be very performant, especially when it is compiled to native code. This makes it a good choice for developing high-performance applications.
  • Enterprise adoption: Java is by far the most popular language of the enterprise. And that's not changing anytime soon.
  • Community: Java has a large and active community of developers. This means that there are plenty of resources available to help you learn and use the language.
  • Tooling: There are a wide variety of tools available for Java development, including IDEs, debuggers, and code quality tools. This makes it easy to develop and debug Java code.
  • Ubiquity: Java is everywhere. Java runs anywhere.

What the future holds for Java

Despite the apparent slowness in adhering to new trends, Java is an actively evolving language. Each new version adds new features and improvements, making Java a powerful and versatile language that is still relevant today.

Those who complain that Java is slow don't understand successful products and enterprise software. This cadence is required when your solutions run on billions of devices, process billions of dollars in financial transactions, and support 5-9's SLA. How simple do you think that is?

Conclusion

In summary, don't let the doomers convince you. Java still matters!

And in case it helps, here's one reason to (re)consider Java: Java is not going away anytime soon. I predict at least another 20 years of strong support for Java. So for those that like a stable language, a great ecosystem, no shortage of work, a great salary and a great career should consider learning and working with the language.

About the Author

Bruno Hildenbrand      
Principal Architect, HildenCo Solutions.