Panzoto

Reading Time: 2 minutes

The Biden administration just released an executive order on October 30th, 2023 regarding the governance of artificial intelligence. I applaud the effort the white house is making towards regulating the AI industry and protecting the general public regarding privacy and and safety concerns. These efforts focuses on national security issues regarding biological, chemical, and nuclear weapons, specifically associated with biotechnology and cybersecurity. It calls on the respective federal agencies to draft plans in the next 90-270 days to regulate companies who are involved in the AI industry.

There will be a lot of good things coming out of this executive order. But as human nature dictates, it’s just easier to criticize than to praise the positives. So here is what I think that needs improvement:

1. My major concern is that this document will prevent the entry of small companies to the large language model (LLM) market. Early in the document, one of the principle for the guideline was to promote competition, especially from the smaller companies. But later in the document, it calls on the agencies to draft guidelines that require companies to report the development LLMs over 10 billion parameters. The intention is to keep track of the models that can be used for many purposes, including those that pose national security risks. However this requirement inadvertently impede the development of models by small companies because they wouldn’t have the ability to generate the documentation required by these agencies. This 10 billion parameter limit means the latest llama open source models is included in the reporting requirements, including companies that simply possessing these models. This mean the making of large models are left in the hands of the tech companies like Google, Facebook, Microsoft, etc. This is not what the general public like to see.
2. Another thing I see is that some of the guidelines and policies are difficult to enforce. For examples, one of the principles is reducing the effect of AI on general workforce. It is a well intended principle. But in the practice, profit seeking nature of companies means that they will use the power of AI to reduce workforce spending. Even though many companies talked about “augmentation” of workers, it’s a fact that the worker count have been reduced because of AI implementations. The reality is that for people who cannot or unwilling to learn about AI technologies, they will be impacted by workforce reduction in the future. And this future could be sooner than people think.
3. It’s late. Regarding privacy laws, we are behind Europe on GDPR requirements. Some states have their privacy laws, such as California, but we still don’t have unified national requirements. For comparison, China started to mandate online commerce companies to stop using consumer data to track and target consumers with the sole purpose of maximizing profit. One example it to prevent the practice of giving new customers better deals while “ignoring” old customers. Or selectively give people worse deals because the companies knows that they will purchase such items regardless.

Reading Time: 4 minutes

Over a month ago, I was back on the job market. I connected with my friends and here are the opportunities I have found and a few tips on what the market is looking for.

1. Products that use search. I met a lot of knowledgable individual in this area and learned a great deal along the way. Needs in this area revolves around using traditional search algorithms and mix in neural network approaches as well. I was surprised to find that keyword-based algorithms like BM25 is still one of the widely used methods. BM25 is similar to TFIDF, but rather than just count the rare words, it is also divide the frequency by the length of the text. So the longer text were not given favor just because there are more words in it. Even though they don’t have semantic meaning, this algorithm is still among the top performers of BEIR scores. I was surprise to find this since I have used similar algorithms in topic modeling, but found the results to be much inferior to using sentence embedding. The group I talked to in this area is more concerned about implementation of personalization of the search result. Many of techniques require “augmentation” of the search query. For examples, using intent classification on the search query to generalize it’s purpose. Since many search query are short phrases that barely have context, the algorithm essentially need to guess what people are asking for based on the personal history, location, and what other users found to be useful. Other preprocessing including segmentation of the search context, so larger groups are identified. The groups are basic units to be search rather than individual documents.
2. Fine-tuning of large language models. Since GPT4 and large language models is the current and hot topic, many companies are looking to take advantage and ride the wave. It’s not always the best solution for their business case, but companies do occasionally find it useful. More often people use out-of-the-box LLMs to gain knowledge to their documentation. For example, if we want to able to search through written documents and find relevant answers, we can simply put that large document in chunks through the GPT4 model and ask for an answer. Or we can use LLM to build embeddings and search query against the embedding using cosine similarity and rank the results. Up until recently, GPT4 models have no access to internet data to fact check. Even if they do, they might not know business specific knowledge. So there are packages help user search through those information as well. Algorithms like Retrieval-Augmented Generation and Low Rank Adaptation are particularly useful. A major advantage of this approach is that they don’t require a lot of data to fine tune the large language model.
3. Causal inference modeling. This type of analysis is often for industries that often need to figure out the root cause. One example could be the insurance industry. For example, if I want to figure out the important factors that influence the price of auto insurance, I can simply run a general linear model with feature like age, marital status, etc. to predict the insurance premium. And use models with feature importance to figure out which feature influence the premium the most. But this often does not solve the business problem since the executives will question why they think a particular feature is important. So the data scientist not only need to figure out the what the factors are, they also need to find out how. One way people do this is by constructing equations they think how these features influence the premium. This is often based on their industry experience and intuitive sense. I cannot say I understand this part, so it seems that many years of experience is need.
4. Company-wise tool development. In larger corporations, there are research teams that build machine learning tools aimed to help the whole company. Since ML talents are rare for any company, they are looking to maximize the value. Instead of targeting specific use cases, they are asked to develop generic tools. This is actually the most common theme I have seen for the jobs I applied to. The needs range from fresh new departments to seasoned corporations with years of ML research experience. The only difference is that newer teams focus more on tool building, whereas mature teams thing more about large scale production and data normalization.
5. Specific product development. As apposed to company-wise tool development, many company have specific productization goals. They may or may not include ML specific component, but I tried for these positions anyways. Most of them are not the best fit for me. Some are because of ethical reasons, and some simply because I’m not that good of a programmer and they don’t need ML knowledge at all. All and all, I’m glad I tried just to see where I’m and what I don’t want to do.

At the end, my decisions is to go with a position that’s most comfortable for family reasons. That same role also have enough ML needs and I can use my past experience and explore new things at the same time. To sum up the learnings for this experience, is to get as many options as possible, and really figure out what I wanted to do. Don’t shy away from tough interview questions because sometimes they help you figure out life purpose as well. And for offer negotiations, having multiple offers make it a lot easier to play the “don’t tell your salary expectation” game. It helped me immensely since I don’t have natural game to get the max offer I can get.

Reading Time: 3 minutes

Expanding on using the Decision Story tool to help people make decisions. I want to explore the decision making process of business executives, and how this tool could help that target audience. By talking to people who are in the executive role and interacted with CEOs, I found one popular way for which CEO uses to make decisions. For a decision that may change the direction of the company, CEOs would like to ask the alternatives, and how these alternative changes the financial bottom line, as well as departments of the company. They want to know what would change if they make the decisions in as many aspect of the company as possible. So they could evaluate if those changes is good for the company overall.

So I though, what if the decision making tool could incorporate those aspect and ask the large language models (LLM) to evaluate it through the lens of an Econ. An Econ is often referred to in the economy academics as someone who always make the most logical decisions with the lack of emotion and bias. I have tried to use this approach and got some reasonable answers from the LLM. And I have found the answer covered some aspect of the decision that I haven’t realized. There are definitely CEOs who are smarter than me, but it probably doesn’t hurt for people to know that they already covered the basics. Here are some of aspected worth mentioning when asking the LLM for alternatives.

Here is an example in which I asked the tool to help me make a decision to whether to use a in-house model, 3rd Party model, or a combination approach to improve an existing product.
Here is what I put in:

“I’m an CEO of a mid size company with 300+ employees. We are working on speech analytics and helping customer to understand call center conversations. Large language models are getting big. I have some decisions about which model to use. We developed some simple binary classification model in house. We can use Openai 3rd party models. Or we can use our existing rule based solution. Or we can use a combination of some of the above. Please provide the pros and cons about each and provide other alternatives for long term growth. Specifically aim at IPO in next few years.”

And here is things I’m putting into the models.

1. Objective Definition:

Define the goal you’re trying to achieve. In your case, it might be “select the best AI approach to enhance our speech analytics service.”

2. Information Gathering:

Gather as much information as possible relevant to your decision. This includes:

• Business Requirements: Specific needs of your business and customers.
• Technical Requirements: Scalability, data volumes, security, etc.
• Market Trends: What your competitors are doing, trends in AI technology.
• Legal and Regulatory Requirements: Compliance to data privacy laws, etc.
• Financials: Budgets, potential ROI, etc.

3. Decision Criteria:

Establish the criteria you will use to evaluate your options. This could include factors like cost, time to implement, ease of use, scalability, potential return on investment, etc.

4. Generate Alternatives:

Generate a list of potential options. In your case, these are the AI technologies you’re considering: In-house model, third-party model, rule-based solution, or a hybrid.

5. Evaluate Alternatives:

Use your decision criteria to evaluate each alternative. This may involve financial modeling, technical assessments, consulting with experts, and other forms of analysis.

6. Select the Best Alternative:

Based on your evaluations, select the option that best meets your criteria and aligns with your business objectives.

7. Action Plan:

Develop an action plan for implementing your decision. This should include key milestones, resources required, risks and mitigation plans, and a timeline.

8. Review Decision:

After a certain period of implementation, review the decision to see if it’s delivering the expected results. If not, understand why and adjust as needed.

Reading Time: < 1 minute

Since Transformers was invented, there have been plenty of large language models as Machine Learning has taken off in the last few years. GPT3, RoBERTa, PaLM, and many more. Most of them were trained by large companies like OpenAI, Facebook, or Google. They often use open-source databases like Wikipedia. The problem with these models is they retain the same bias that already exists in these data since humans are not free of bias. The companies may take precautions when deploying the final results of the models, but the training data were not examined for bias. In addition, because the dominating language in the business world that make money is still English, not much attempt was made in including low-resource languages.

BLOOM, the BigScience Large Open-science Open-access Multilingual Language Model, was made by scientists for research purposes. They specifically focused on eliminating bias in the training data. And also included many more Asian and African languages. These improvements target the research and education sector. So far, its direct usage is language generation, which generally still suffers from repetition. But the encoding itself can be used for summarization and question and answer.

Huggingface model card: Link here

Research site: Link Here