百度360必应搜狗淘宝本站头条
当前位置:网站首页 > 热门文章 > 正文

golang 深入分析 http.RoundTripper

bigegpt 2024-08-24 23:10 2 浏览

1、RoundTripper 接口

// RoundTripper is an interface representing the ability to execute a
// single HTTP transaction, obtaining the Response for a given Request.
//
// A RoundTripper must be safe for concurrent use by multiple
// goroutines.
type RoundTripper interface {
   // RoundTrip executes a single HTTP transaction, returning
   // a Response for the provided Request.
   //
   // RoundTrip should not attempt to interpret the response. In
   // particular, RoundTrip must return err == nil if it obtained
   // a response, regardless of the response's HTTP status code.
   // A non-nil err should be reserved for failure to obtain a
   // response. Similarly, RoundTrip should not attempt to
   // handle higher-level protocol details such as redirects,
   // authentication, or cookies.
   //
   // RoundTrip should not modify the request, except for
   // consuming and closing the Request's Body. RoundTrip may
   // read fields of the request in a separate goroutine. Callers
   // should not mutate or reuse the request until the Response's
   // Body has been closed.
   //
   // RoundTrip must always close the body, including on errors,
   // but depending on the implementation may do so in a separate
   // goroutine even after RoundTrip returns. This means that
   // callers wanting to reuse the body for subsequent requests
   // must arrange to wait for the Close call before doing so.
   //
   // The Request's URL and Header fields must be initialized.
   RoundTrip(*Request) (*Response, error)
}

一句话概括:可以把RoundTripper看成是 http.Client 的中间件

2、场景

(1)缓存http responses。如果缓存中存在,则直接从缓存中取。如果缓存中不存在,则重新请求

(2)根据需要适当的设置HTTP headers

(3)Rate limiting

3、实例

使用HTTP.roundtripper的实现来缓存HTTP响应

(1)实现一个 HTTP Server

package main

import (
   "fmt"
   "net/http"
)

func main() {

   mux := http.NewServeMux()

   mux.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
      //This is here so wen can actually see what the responses that have been cached don't get here
      fmt.Println("The request actually got here")
      writer.Write([]byte("You got here"))
   })
   http.ListenAndServe(":8080", mux)
}

(2)新建一个http.Transport并且实现 http.RoundTripper接口,同时让其具有缓存功能

package cacheTransport

import (
   "bufio"
   "bytes"
   "errors"
   "fmt"
   "net"
   "net/http"
   "net/http/httputil"
   "sync"
   "time"
)

func cachKey(r *http.Request) string {
   return r.URL.String()
}

type CacheTransport struct {
   data              map[string]string
   mu                sync.RWMutex
   originalTransport http.RoundTripper
}

func NewCacheTransport() *CacheTransport {
   return &CacheTransport{
      data: make(map[string]string, 20),
      originalTransport: &http.Transport{
         DialContext: (&net.Dialer{
            Timeout:   2 * time.Second,
            KeepAlive: 30 * time.Second,
            DualStack: true,
         }).DialContext,
         ResponseHeaderTimeout: 5 * time.Second,
         MaxIdleConns:          100,
         IdleConnTimeout:       90 * time.Second,
         TLSHandshakeTimeout:   10 * time.Second,
      }}
}

func (c *CacheTransport) RoundTrip(r *http.Request) (*http.Response, error) {
   //Check if we have the respose cached...
   // if yes,we don't have to hit the server
   // we just return it as is from the cache store.
   if val, err := c.Get(r); err == nil {
      fmt.Println("Fetching the response from the cache")
      return cachedResponse([]byte(val), r)
   }

   // we don't have the response cached,the store was probably cleared.
   // Make the request to the server
   resp, err := c.originalTransport.RoundTrip(r)
   if err != nil {
      return nil, err
   }

   // Get the body of the response so we can save it int the cache for the next request.
   buf, err := httputil.DumpResponse(resp, true)
   if err != nil {
      return nil, err
   }
   //Saving it to the store
   c.Set(r, string(buf))

   fmt.Println("Fetching the data form the real source")
   return resp, nil
}

func cachedResponse(b []byte, r *http.Request) (*http.Response, error) {
   buf := bytes.NewBuffer(b)
   return http.ReadResponse(bufio.NewReader(buf), r)
}
func (c *CacheTransport) Set(r *http.Request, value string) {
   c.mu.Lock()
   defer c.mu.Unlock()
   c.data[cachKey(r)] = value
}

func (c *CacheTransport) Get(r *http.Request) (string, error) {
   c.mu.Lock()
   defer c.mu.Unlock()

   if val, ok := c.data[cachKey(r)]; ok {
      return val, nil
   }

   return "", errors.New("key not found in cache")
}

func (c *CacheTransport) Clear() error {
   c.mu.Lock()
   c.mu.Unlock()
   c.data = make(map[string]string)
   return nil
}

(3)实现客户端的引导程序main函数,同时具有一个timer来定时清理缓存

package main

import (
   "awesomeProject/interview/roundtripper/cacheTransport"
   "fmt"
   "io/ioutil"
   "log"
   "net/http"
   "os"
   "os/signal"
   "strings"
   "syscall"
   "time"
)

func main() {

   cacheTransport := cacheTransport.NewCacheTransport()

   // Create a custom client so we can make use of our RoundTripper
   // If you make use of http.Get(),the default http client located at http.DefaultClient is used instead
   // Since we have special needs,we have to make use of our own http.RoundTripper implementation
   client := &http.Client{
      Transport: cacheTransport,
      Timeout:   time.Second * 5,
   }

   // Time to clear the cache store so we can make requet to the original server rather than fetch from the cache store
   // This is to replicate real expiration of data in a cache store
   cacheClearTicker := time.NewTicker(time.Second * 5)

   // Make a new request every second
   // This would help demonstrate if the response if coming from the real server or the cache
   reqTicker := time.NewTicker(time.Second * 1)

   terminateChannel := make(chan os.Signal, 1)

   signal.Notify(terminateChannel, syscall.SIGTERM, syscall.SIGHUP)

   req, err := http.NewRequest(http.MethodGet, "http://localhost:8080", strings.NewReader(""))
   if err != nil {
      panic("Whoops")
   }

   for {
      select {
      case <-cacheClearTicker.C:
         // Clear the cache so we can hit the original server
         cacheTransport.Clear()
      case <-terminateChannel:
         cacheClearTicker.Stop()
         reqTicker.Stop()
         return
      case <-reqTicker.C:

         resp, err := client.Do(req)
         if err != nil {
            log.Printf("An error occurred.... %v", err)
            continue
         }

         buf, err := ioutil.ReadAll(resp.Body)
         if err != nil {
            log.Printf("An error occurred...%v", err)
            continue
         }
         fmt.Printf("The body of the response if \"%s\" \n\n", string(buf))
      }
   }
}

(4)服务端执行结果如下

客户端执行结果如下:




相关推荐

AI「自我复制」能力曝光,RepliBench警示:大模型正在学会伪造身份

科幻中AI自我复制失控场景,正成为现实世界严肃的研究课题。英国AISI推出RepliBench基准,分解并评估AI自主复制所需的四大核心能力。测试显示,当前AI尚不具备完全自主复制能力,但在获取资源...

【Python第三方库安装】介绍8种情况,这里最全看这里就够了!

**本图文作品主要解决CMD或pycharm终端下载安装第三方库可能出错的问题**本作品介绍了8种安装方法,这里最全的python第三方库安装教程,简单易上手,满满干货!希望大家能愉快地写代码,而不要...

pyvips,一个神奇的 Python 库!(pythonvip视频)

大家好,今天为大家分享一个神奇的Python库-pyvips。在图像处理领域,高效和快速的图像处理工具对于开发者来说至关重要。pyvips是一个强大的Python库,基于libvips...

mac 安装tesseract、pytesseract以及简单使用

一.tesseract-OCR的介绍1.tesseract-OCR是一个开源的OCR引擎,能识别100多种语言,专门用于对图片文字进行识别,并获取文本。但是它的缺点是对手写的识别能力比较差。2.用te...

实测o3/o4-mini:3分钟解决欧拉问题,OpenAI最强模型名副其实!

号称“OpenAI迄今为止最强模型”,o3/o4-mini真实能力究竟如何?就在发布后的几小时内,网友们的第一波实测已新鲜出炉。最强推理模型o3,即使遇上首位全职提示词工程师RileyGoodsid...

使用Python将图片转换为字符画并保存到文件

字符画(ASCIIArt)是将图片转换为由字符组成的艺术作品。利用Python,我们可以轻松实现图片转字符画的功能。本教程将带你一步步实现这个功能,并详细解释每一步的代码和实现原理。环境准备首先,你...

5分钟-python包管理器pip安装(python pip安装包)

pip是一个现代的,通用、普遍的Python包管理工具。提供了对Python包的查找、下载、安装、卸载的功能,是Python开发的基础。第一步:PC端打开网址:选择gz后缀的文件下载第二步:...

网络问题快速排查,你也能当好自己家的网络攻城狮

前面写了一篇关于网络基础和常见故障排查的,只列举了工具。没具体排查方式。这篇重点把几个常用工具的组合讲解一下。先有请今天的主角:nslookup及dig,traceroute,httping,teln...

终于把TCP/IP 协议讲的明明白白了,再也不怕被问三次握手了

文:涤生_Woo下周就开始和大家成体系的讲hadoop了,里面的每一个模块的技术细节我都会涉及到,希望大家会喜欢。当然了你也可以评论或者留言自己喜欢的技术,还是那句话,希望咱们一起进步。今天周五,讲讲...

记一次工控触摸屏故障的处理(工控触摸屏维修)

先说明一下,虽然我是自动化专业毕业,但已经很多年不从事现场一线的工控工作了。但自己在单位做的工作也牵涉到信息化与自动化的整合,所以平时也略有关注。上一周一个朋友接到一个活,一家光伏企业用于启动机组的触...

19、90秒快速“读懂”路由、交换命令行基础

命令行视图VRP分层的命令结构定义了很多命令行视图,每条命令只能在特定的视图中执行。本例介绍了常见的命令行视图。每个命令都注册在一个或多个命令视图下,用户只有先进入这个命令所在的视图,才能运行相应的命...

摄像头没图像的几个检查方法(摄像头没图像怎么修复)

背景描述:安防监控项目上,用户的摄像头运行了一段时间有部分摄像头不能进行预览,需要针对不能预览的摄像头进行排查,下面列出几个常见的排查方法。问题解决:一般情况为网络、供电、设备配置等情况。一,网络检查...

小谈:必需脂肪酸(必需脂肪酸主要包括)

必需脂肪酸是指机体生命活动必不可少,但机体自身又不能合成,必需由食物供给的多不饱和脂肪酸(PUFA)。必需脂肪酸主要包括两种,一种是ω-3系列的α-亚麻酸(18:3),一种是ω-6系列的亚油酸(18:...

期刊推荐:15本sci四区易发表的机械类期刊

  虽然,Sci四区期刊相比收录在sci一区、二区、三区的期刊来说要求不是那么高,投稿起来也相对容易一些。但,sci四区所收录的期刊中每本期刊的投稿难易程度也是不一样的。为方便大家投稿,本文给大家推荐...

be sick of 用法考察(be in lack of的用法)

besick表示病了,做谓语.本身是形容词,有多种意思.最通常的是:生病,恶心,呕吐,不适,晕,厌烦,无法忍受asickchild生病的孩子Hermother'sverysi...